-- Logs begin at Fri 2022-10-28 08:59:30 UTC, end at Fri 2022-10-28 10:02:19 UTC. -- Oct 28 09:36:07.742668 dv-cinder-zed-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 28 09:36:07.752543 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 28 09:36:07.754145 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Oct 28 09:36:07 2022] *** Oct 28 09:36:07.754145 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 28 09:36:07.754145 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 28 09:36:07.754145 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: nodename: dv-cinder-zed-3-iscsi Oct 28 09:36:07.754145 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: machine: x86_64 Oct 28 09:36:07.754145 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: clock source: unix Oct 28 09:36:07.754145 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: pcre jit disabled Oct 28 09:36:07.754145 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: detected number of CPU cores: 4 Oct 28 09:36:07.754145 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: current working directory: / Oct 28 09:36:07.754145 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: detected binary path: /usr/bin/uwsgi-core Oct 28 09:36:07.754145 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: your processes number limit is 47680 Oct 28 09:36:07.754996 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: your memory page size is 4096 bytes Oct 28 09:36:07.754996 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: detected max file descriptor number: 2048 Oct 28 09:36:07.754996 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: lock engine: pthread robust mutexes Oct 28 09:36:07.754996 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: thunder lock: enabled Oct 28 09:36:07.754996 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 28 09:36:07.754996 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 28 09:36:07.794935 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: Python main interpreter initialized at 0x5563cc1d79c0 Oct 28 09:36:07.794935 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: python threads support enabled Oct 28 09:36:07.794935 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: your server socket listen backlog is limited to 100 connections Oct 28 09:36:07.794935 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: your mercy for graceful operations on workers is 90 seconds Oct 28 09:36:07.795583 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: mapped 403077 bytes (393 KB) for 2 cores Oct 28 09:36:07.795583 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: *** Operational MODE: preforking *** Oct 28 09:36:07.795836 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: *** uWSGI is running in multiple interpreter mode *** Oct 28 09:36:07.795836 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: spawned uWSGI master process (pid: 112049) Oct 28 09:36:07.795909 dv-cinder-zed-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 28 09:36:07.798494 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: spawned uWSGI worker 1 (pid: 112050, cores: 1) Oct 28 09:36:07.798494 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: spawned uWSGI worker 2 (pid: 112051, cores: 1) Oct 28 09:36:07.798494 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 28 09:36:08.488741 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 28 09:36:08.496684 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 28 09:36:09.587130 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 09:36:09.587130 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: from cryptography.utils import int_from_bytes Oct 28 09:36:09.587769 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 09:36:09.587769 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: from cryptography.utils import int_from_bytes Oct 28 09:36:09.587905 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 09:36:09.587905 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: from cryptography.utils import int_from_bytes Oct 28 09:36:09.588286 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 09:36:09.588286 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: from cryptography.utils import int_from_bytes Oct 28 09:36:12.130313 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 28 09:36:12.131044 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 28 09:36:12.131208 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 28 09:36:12.134574 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 28 09:36:12.134734 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 28 09:36:12.134895 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 28 09:36:12.134895 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 28 09:36:12.135333 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 28 09:36:12.135477 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.135608 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 28 09:36:12.135608 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 28 09:36:12.135859 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.135859 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.136113 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.136242 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.136377 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.136507 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.136634 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.136768 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.136894 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.137022 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.137178 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.137327 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.137473 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.137593 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.137723 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.137850 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.137979 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.138108 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.138297 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.138498 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-zed-3-iscsi {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.138675 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.138851 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.139015 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.139183 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.139362 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-zed-3-iscsi {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.139531 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.139702 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.140014 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.140205 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.140344 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.140493 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.140628 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.140767 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.140767 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.141027 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.141175 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.141302 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.141459 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.141594 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.141747 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.141909 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.142043 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.142180 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.142316 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.142471 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.142621 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.142777 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.142914 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.143060 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.143259 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.143414 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.143553 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.143702 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.143843 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.143978 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.144125 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.144260 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.144412 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.144555 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-zed-3-iscsi {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.144700 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.144830 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.144983 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-zed-3-iscsi {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.145124 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.145279 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.145417 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.145566 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.145712 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.145858 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.146007 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.146138 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.146275 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.146412 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.146555 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.146684 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.146804 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.146939 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.147067 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.147199 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.147345 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.147485 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.147644 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.147783 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.147903 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.148036 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.148157 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.148277 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.148406 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.148527 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.148674 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.148794 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.148915 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.149041 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.149176 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.149304 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.149423 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.149542 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.149672 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.149837 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.150001 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.150142 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.150262 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.150382 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.150527 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.150646 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.150784 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.150909 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.151028 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.151159 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.151485 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.151704 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.151845 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.152044 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.152210 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.152344 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.152492 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.152615 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.152743 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.152871 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.152990 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.153118 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.153256 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.153377 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.153553 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.153696 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.153825 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.153952 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.154085 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.154221 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.154348 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.154509 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.154633 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.154818 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.154948 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.155078 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.155197 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.155318 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.155455 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.155590 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.155718 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.155843 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.155968 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.156089 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.156216 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.156351 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.156471 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.156590 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.156717 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.156836 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.156964 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.157112 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.157233 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.157354 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.157482 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.157601 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.157746 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.157925 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.158052 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.158172 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.158300 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.158499 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.158651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.159292 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.159426 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.159550 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.159670 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.159800 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.159929 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.160060 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.160203 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.160331 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.160451 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.160585 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.160715 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.160842 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.160965 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.161101 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.161222 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.161350 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.161487 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.161856 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.161985 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.162113 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.162249 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.162382 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.162533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.162671 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.162798 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.162928 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.163057 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.163190 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.163310 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.163442 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.163561 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.163708 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.163844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.163978 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.164106 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.164233 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.164359 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.164478 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.164607 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.164744 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.164881 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.165124 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.165245 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.165373 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.165510 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.165639 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.165766 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.165886 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.166006 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.166125 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.166260 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.166492 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.166638 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.166761 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.166891 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.167020 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.167140 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.167271 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.167391 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.167511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.167641 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.167769 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.167889 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.168076 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.168197 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.168324 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.168443 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.168570 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.168699 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.168826 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.168946 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.169075 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.169195 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.169315 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.169448 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.169570 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.169700 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.169830 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.169951 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.170072 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.170202 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.170364 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.170534 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.170655 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.170777 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.170902 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.171037 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.171158 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.171278 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.171398 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.171519 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.171659 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.171779 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.171910 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.172038 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.172175 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.172295 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.172431 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.172551 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.172673 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.172805 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.172934 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.173056 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.173203 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.173332 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.173454 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.173584 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.173707 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.173888 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.174035 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.174166 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.174302 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.174496 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.174635 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:36:12.174778 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.174911 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.175116 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.175292 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.175414 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.175542 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.175749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.175872 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.176012 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.176134 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.176338 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.176479 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.176654 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.176778 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.176970 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.177092 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.177282 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.177423 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.177593 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.177715 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.177895 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.178025 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.178200 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.178334 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.178531 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.178656 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.178844 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.178980 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.179159 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.179283 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.179474 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.179611 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.179789 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.179911 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.180098 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.180235 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.180407 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.180541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.180706 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.180828 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.181019 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.181166 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.181340 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.181464 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.181654 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.181790 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.181957 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.182092 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.182291 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.182482 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.182650 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.182774 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.182975 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.183115 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.183295 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.183420 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.183610 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.183744 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.183919 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.184048 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.184221 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.184344 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.184525 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.184659 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.184856 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.184981 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.185167 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.185342 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.185522 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.185643 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.185788 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.185908 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.186063 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.186186 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.186374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.186530 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.186702 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.186837 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.187012 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.187145 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.187317 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.187462 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.187624 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.187758 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.187927 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.188059 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.188261 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.188417 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.188594 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.188719 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.188855 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.188982 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.189164 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.189300 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.189476 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.189608 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.189786 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.189908 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.190647 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.190792 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.190958 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.191081 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.191262 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.191389 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.191561 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.191696 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.191862 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.191996 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.192162 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.192287 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.192464 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.192596 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.192761 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.192884 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.193078 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.193213 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.193397 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.193520 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.193697 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.193822 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.194026 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.194212 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.194347 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.194512 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.194632 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.194757 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.194934 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.195074 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.195243 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.195380 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.195524 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.195649 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.195830 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.195979 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.196134 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.196258 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.196432 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.196570 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.196750 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.196886 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.197053 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.197185 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.197353 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.197486 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.197666 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.197810 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.197966 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.198088 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.198270 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.198479 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.198603 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.198827 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.198949 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.199086 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.199217 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.199352 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.199489 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.199610 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.199797 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.199921 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.200106 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.200230 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.200413 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.200537 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.200722 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.200851 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.201048 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.201186 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.201351 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.201475 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.201660 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.201782 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.201980 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.202122 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.202293 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.202442 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.202624 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.202749 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.202923 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.203065 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.203225 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.203359 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.203521 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.203644 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.203822 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.203959 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.204125 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.204306 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.204440 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.204580 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.204702 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.204840 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.205008 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.205135 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.205307 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.205489 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.205645 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.205871 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.206002 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.206127 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.206255 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.206378 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.206626 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.206759 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.206881 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.207021 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.207153 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.207285 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.207420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.207540 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.207759 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.207884 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.208060 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.208194 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.208399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.208589 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.208734 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.208859 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.209049 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.209187 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.209360 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.209481 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.209664 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.209785 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.209974 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.210108 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.210276 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.210422 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.210607 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.210728 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.210915 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.211052 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.211224 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.211357 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.211535 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.211657 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.211838 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.211972 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.212151 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.212274 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.212455 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.212578 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.212771 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.212905 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.213075 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.213208 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.213390 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.213515 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.213686 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.213807 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.213997 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.214122 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.214337 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.214549 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.214686 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.214817 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.214978 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.215112 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.215286 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.215417 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.215597 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.215726 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.215906 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.216058 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.216206 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.216327 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.216513 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.216635 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.216817 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.216940 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.217122 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.217250 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.217428 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.217551 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.217732 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.217854 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.218040 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.218174 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.218343 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.218488 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.218714 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.218856 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.219032 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.219176 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.219338 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.219462 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.219652 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.219774 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.219984 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.220204 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.220336 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.220587 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.220710 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.220911 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.221042 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.221215 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.221829 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.222033 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.222155 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.222326 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.222511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.222672 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.222798 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.222978 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.223109 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.223283 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.223404 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.223586 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.223718 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.223885 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.224008 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.224193 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.224327 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.224512 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.224684 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.224806 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.224931 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.225108 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.225250 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.225409 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.225532 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.225715 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.225854 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.226015 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.226143 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.226315 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.226465 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.226643 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.226811 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.226947 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.227071 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.227256 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.227389 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.227564 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.227688 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.227894 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.228031 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.228178 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.228312 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.228477 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.228598 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.228783 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.228954 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.229076 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.229202 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.229380 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.229516 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.229681 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.229803 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.229983 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.230107 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.230282 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.230485 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.230638 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.230777 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.230946 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.231344 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.231552 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.231685 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.231825 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.231949 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.232070 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.232206 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.232329 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.232463 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.232595 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.232726 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.232857 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.232978 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.233114 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.233237 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.233361 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.233499 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.233620 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.233743 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.233910 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.234043 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.234217 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.234350 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.234547 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.234669 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.234872 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.235006 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.235158 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.235282 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.235476 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.235600 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.235784 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.235964 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.236097 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.236289 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.236420 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.236610 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.236744 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.236981 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.237302 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.237423 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.237619 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.237763 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.237949 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.238190 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.238315 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.238547 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.238672 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.238863 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.239065 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.239189 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.239318 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.239515 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.239652 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.239833 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.239969 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.240161 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.240285 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.240473 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.240597 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.240785 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.240909 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.241089 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.241217 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.241393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.241523 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.241697 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.241821 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.241995 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.242134 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.242299 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.242446 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.242653 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.242786 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.242961 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.243085 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.243274 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.243399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.243581 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.243707 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.243877 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.243999 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.244181 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.244305 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.244482 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.244623 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.244786 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.244976 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.245098 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.245223 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.245384 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.245522 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.245692 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.245816 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.245993 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.246122 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.246298 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.246519 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.246642 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.246769 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.246895 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.247017 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.247200 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.247336 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.247501 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.247637 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.247839 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.247963 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.248141 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.248264 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.248475 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.248599 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.248791 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.248914 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.249119 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.249274 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.249419 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.249554 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.249720 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.249857 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.250024 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.250156 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.250321 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.250495 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.250659 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.250808 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.250966 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.251091 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.251270 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.251405 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.251575 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.251794 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.251916 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.252041 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.252230 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.252372 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.253000 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.253157 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.253291 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.253412 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.253560 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.253695 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.253859 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.254003 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.254163 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.254296 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.254483 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.254607 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.254786 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.254908 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.255109 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.255252 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.255387 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.255510 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.255656 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.255788 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.255959 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.256093 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.256284 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.256416 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.256612 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.256873 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.256998 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.257138 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.257263 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.257403 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.257540 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.257704 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.257840 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.258006 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.258145 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.258307 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.258540 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.258678 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.258828 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.258950 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.259087 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.259209 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.259335 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.259524 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.259651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.259829 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.259972 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.260128 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.260262 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.260427 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.260550 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.260730 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.260870 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.261033 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.261158 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.261332 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.261469 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.261636 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.261769 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.261946 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.262069 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.262251 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.262375 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.262555 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.262677 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.262865 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.262995 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.263165 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.263288 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.263466 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.263601 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.263763 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.263886 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.264063 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.264187 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.264358 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.264483 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.264658 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.264780 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.264953 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.265090 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.265278 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.265410 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.265544 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.265668 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.265848 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.265986 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.266146 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.266271 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.266487 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.266650 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.266786 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.266921 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.267080 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.267269 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.267418 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.267545 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.267716 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.267852 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.268041 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.268219 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.268419 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.268543 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.268734 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.268858 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.269050 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.269175 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.269362 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.269484 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.269668 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.269795 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.269969 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.270092 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.270267 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.270388 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.270609 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.270758 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.270913 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.271037 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.271226 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.271396 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.271531 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.271653 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.271819 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.271943 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.272134 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.272257 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.272444 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.272569 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.272753 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.272878 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.273054 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.273196 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.273371 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.273499 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.273681 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.273805 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.273979 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.274113 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.274284 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.274431 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.274624 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.274747 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.274939 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.275076 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.275256 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.275439 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.275561 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.275698 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.275858 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.275996 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.276167 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.276292 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.276478 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.276634 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.276767 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.276898 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.277066 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.277190 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.277374 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.277499 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.277643 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.277773 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.277947 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.278087 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.278248 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.278382 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.278568 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.278699 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.278877 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.279013 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.279178 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.279303 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.279486 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.279673 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.279923 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.280059 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.280238 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.280446 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.280567 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.280693 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.280840 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.281047 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.281183 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.281409 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.281535 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.281746 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.281871 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.282063 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.282209 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.282358 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.282568 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.282796 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.282918 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.283114 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.283237 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.283426 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.283560 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.283758 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.284389 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.284569 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.284712 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.284875 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.285008 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.285184 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.285307 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.285544 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.285717 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.285879 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.286044 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.286196 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.286358 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.286811 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.287001 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.287125 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.287252 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.287385 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.287562 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.287683 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.287821 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.287946 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.288071 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.288197 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.288326 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.288461 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.288583 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.288765 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.288949 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.289147 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.289271 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.289460 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.289609 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.289790 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.289910 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.290156 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.290313 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.290504 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.290628 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.290812 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.290934 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.291126 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.291249 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.291443 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.291577 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.291779 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.291901 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.292037 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.292160 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.292329 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.292455 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.292636 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.292769 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.292945 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.293069 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.293258 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.293393 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.293564 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.293699 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.293875 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.294009 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.294177 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.294313 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.294501 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.294625 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.294824 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.294947 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.295134 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.295274 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.295453 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.295603 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.295855 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.295986 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.296164 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.296313 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.296468 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.296591 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.296778 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.296901 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.297090 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.297212 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.297379 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.297504 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.297688 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.297810 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.297971 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.298094 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.298272 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.298419 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.298610 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.298734 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.298924 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.299051 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.299228 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.299352 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.299590 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.299713 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.299906 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.300029 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.300190 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.300315 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.300493 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.300629 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.300793 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.300928 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.301112 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.301237 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.301413 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.301549 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.301734 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.301858 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.302041 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.302164 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.302356 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.302533 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.302691 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.302818 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.302989 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.303113 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.303299 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.303436 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.303603 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.303729 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.303903 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.304026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.304204 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.304329 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.304504 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.304628 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.304842 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.304969 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.305143 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.305267 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.305459 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.305584 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.305785 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.305938 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.306114 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.306234 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.306440 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.306581 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.306752 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.306876 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.307080 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.307256 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.307391 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.307514 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.307648 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.307845 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.307977 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.308102 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.308233 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.308367 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.308539 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.308719 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.308844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.309024 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.309158 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.309337 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.309476 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.309638 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.309772 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.309945 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.310128 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.310256 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.310381 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.310611 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.310743 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.310867 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.310992 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.311125 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.311265 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.311419 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.311545 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.311742 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.311886 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.312041 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.312180 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.312358 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.312495 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.312663 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.312797 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.312964 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.313104 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.313266 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.313390 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.313549 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.313669 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.313886 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.314022 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.314186 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.314311 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.314536 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.314661 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.314844 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.315491 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.315686 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.315810 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.315999 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.316123 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.316337 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.316480 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.316609 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.316733 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.316915 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.317051 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.317234 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.317421 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.317544 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.317670 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.317836 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.318108 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.318252 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.318463 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.318592 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.318769 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.318892 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.319080 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.319222 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.319394 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.319633 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.319758 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.319957 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.320094 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.320276 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.320441 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.320566 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.320691 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.320873 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.321084 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.321208 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.321346 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.321468 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.321591 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.321769 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.321894 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.322063 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.322197 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.322356 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.322513 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.322688 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.322823 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.323012 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.323155 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.323412 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.323564 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.323704 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.323826 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.323951 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.324075 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.324263 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.324400 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.324574 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.324698 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.324883 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.325007 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.325192 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.325333 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.325505 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.325628 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.325818 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.325942 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.326130 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.326265 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.326467 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.326595 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.326776 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.326922 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.327066 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.327188 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.327382 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.327538 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.327688 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.327824 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.327999 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.328178 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.328311 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.328435 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.328602 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.328740 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.328915 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.329038 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.329226 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.329349 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.329536 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.329672 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.329849 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.329983 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.330177 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.330428 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.330567 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.330772 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.330896 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.331112 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.331237 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.331438 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.331586 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.331763 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.331992 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.332117 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.332308 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.332431 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.332639 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.332773 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.332961 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.333085 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.333277 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.333400 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.333587 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.333721 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.333896 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.334031 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.334202 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.334329 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.334515 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.334645 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.334820 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.334946 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.335128 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.335264 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.335435 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.335558 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.335738 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.335859 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.336058 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.336274 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.336410 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.336636 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.336783 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.336956 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.337115 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.337278 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.337438 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.337635 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.337788 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.337941 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.338116 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.338302 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.338451 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.338589 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.338720 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.338843 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.339043 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.339180 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.339359 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.339484 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.339664 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.339788 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.339984 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.340109 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.340288 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.340415 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.340601 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.340725 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.340893 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.341017 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.341200 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.341328 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.341499 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.341634 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.341801 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.341963 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.342087 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.342211 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.342441 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.342593 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.342716 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.342841 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.343009 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.343144 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.343323 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.343458 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.343629 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.343756 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.343934 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.344059 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.344239 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.344377 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.344542 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.344667 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.344855 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.344988 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.345161 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.345300 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.345464 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.345597 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.345766 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.345900 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.346065 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.346190 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.346927 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.347069 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.347231 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.347355 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.347540 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.347663 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.347852 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.347988 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.348158 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.348308 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.348480 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.348665 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.348789 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.348915 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.349091 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.349227 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.349472 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.349649 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.349802 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.349954 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.350106 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.350346 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.350506 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.350632 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.350812 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.350947 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.351072 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.351216 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.351347 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.351495 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.351622 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.351840 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.351989 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.352137 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.352260 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.352506 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.352698 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.352823 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.353029 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.353165 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.353348 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.353474 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.353682 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.353809 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.354011 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.354135 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.354315 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.354466 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.354656 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.354779 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.354957 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.355088 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.355266 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.355390 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.355573 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.355697 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.355882 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.356006 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.356185 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.356318 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.356492 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.356629 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.356799 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.356926 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.357127 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.357259 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.357386 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.357522 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.357693 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.357820 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.357996 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.358121 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.358301 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.358478 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.358710 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.358847 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.358984 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.359127 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.359262 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.359385 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.359559 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.359749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.359885 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.360020 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.360194 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.360354 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.360500 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.360647 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.360784 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.360907 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.361096 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.361230 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.361415 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.361592 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.361747 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.361908 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.362122 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.362250 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.362457 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.362585 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.362882 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.363183 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.363307 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.363505 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.363632 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.363814 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.363950 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.364121 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.364255 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.364437 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.364764 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.364887 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.365090 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.365217 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.365403 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.365528 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.365728 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.365853 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.366042 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.366267 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.366441 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.366726 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.366849 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.367045 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.367239 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.367387 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.367511 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.367648 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.367788 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.367956 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.368081 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.368259 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.368382 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.368571 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.368695 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.368897 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.369086 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.369210 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.369340 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.369500 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.369625 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.369820 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.369945 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.370088 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.370211 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.370378 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.370536 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.370708 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.370832 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.371019 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.371151 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.371316 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.371454 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.371616 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.371739 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.371917 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.372039 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.372218 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.372357 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.372516 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.372649 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.372815 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.372952 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.373116 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.373253 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.373417 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.373541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.373712 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.373846 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.374014 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.374153 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.374316 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.374489 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.374640 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.374765 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.374941 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.375068 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.375240 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.375364 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.375542 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.375678 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.375842 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.375966 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.376143 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.376269 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.376442 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.376568 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.376742 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.376879 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.377042 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.377167 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.377374 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.378034 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.378230 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.378365 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.378574 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.378701 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.378874 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.379015 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.379206 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.379339 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.379498 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.379622 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.379806 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.380062 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.380188 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.380365 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.380498 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.380622 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.380759 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.380894 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.381027 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.381152 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.381288 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.381413 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.381612 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.381749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.381919 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.382056 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.382223 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.382347 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.382553 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.382678 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.382860 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.382987 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.383166 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.383293 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.383491 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.383622 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.383802 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.383928 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.384101 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.384237 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.384421 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.384557 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.384728 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.384863 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.385039 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.385180 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.385350 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.385486 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.385655 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.385780 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.385966 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.386105 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.386270 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.386426 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.386601 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.386741 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.386908 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.387046 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.387215 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.387341 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.387527 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.387670 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.387833 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.387958 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.388138 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.388275 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.388444 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.388578 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.388759 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.388889 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.389074 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.389198 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.389400 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.389538 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.389720 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.389845 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.390034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.390199 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.390349 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.390523 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.390685 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.390818 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.390995 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.391131 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.391255 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=112051) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 28 09:36:12.391392 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.391645 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.391970 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.392282 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.392591 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.392893 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:36:12.393175 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=112050) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 28 09:36:12.547343 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO dbcounter [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Registered counter for database nova_api Oct 28 09:36:12.548768 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO dbcounter [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] Registered counter for database nova_api Oct 28 09:36:12.569924 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_db.sqlalchemy.engines [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112051) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:36:12.572724 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG oslo_db.sqlalchemy.engines [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112050) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:36:12.577602 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writer thread running {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:36:12.581270 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG dbcounter [-] [112050] Writer thread running {{(pid=112050) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:36:12.638812 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.context [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),7c09c1b6-7514-40b3-bb3f-05d78565b59d(cell1) {{(pid=112051) load_cells /opt/stack/nova/nova/context.py:464}} Oct 28 09:36:12.641184 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.context [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),7c09c1b6-7514-40b3-bb3f-05d78565b59d(cell1) {{(pid=112050) load_cells /opt/stack/nova/nova/context.py:464}} Oct 28 09:36:12.642954 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:36:12.643985 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:36:12.645739 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:36:12.645911 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG oslo_concurrency.lockutils [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112050) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:36:12.646268 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG oslo_concurrency.lockutils [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112050) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:36:12.646673 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:36:12.647128 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:36:12.647935 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG oslo_concurrency.lockutils [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=112050) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:36:12.648715 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG oslo_concurrency.lockutils [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112050) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:36:12.649162 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG oslo_concurrency.lockutils [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112050) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:36:12.649266 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:36:12.651249 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG oslo_concurrency.lockutils [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=112050) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:36:12.651375 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO dbcounter [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Registered counter for database nova_cell0 Oct 28 09:36:12.653463 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO dbcounter [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] Registered counter for database nova_cell0 Oct 28 09:36:12.655955 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO dbcounter [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Registered counter for database nova_cell1 Oct 28 09:36:12.658135 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO dbcounter [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] Registered counter for database nova_cell1 Oct 28 09:36:12.665925 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_db.sqlalchemy.engines [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112051) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:36:12.666751 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_db.sqlalchemy.engines [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112051) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:36:12.669356 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG oslo_db.sqlalchemy.engines [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112050) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:36:12.671174 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG oslo_db.sqlalchemy.engines [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112050) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:36:12.673421 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writer thread running {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:36:12.676709 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writer thread running {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:36:12.679808 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG dbcounter [-] [112050] Writer thread running {{(pid=112050) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:36:12.680137 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG dbcounter [-] [112050] Writer thread running {{(pid=112050) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:36:12.824071 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO dbcounter [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Registered counter for database nova_cell0 Oct 28 09:36:12.827154 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO dbcounter [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] Registered counter for database nova_cell0 Oct 28 09:36:12.832616 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_db.sqlalchemy.engines [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112051) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:36:12.837261 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG oslo_db.sqlalchemy.engines [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112050) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:36:12.837848 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writer thread running {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:36:12.845140 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG dbcounter [-] [112050] Writer thread running {{(pid=112050) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:36:13.184903 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: WARNING keystonemiddleware.auth_token [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 09:36:13.191568 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: WARNING keystonemiddleware.auth_token [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 09:36:13.192935 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: WARNING keystonemiddleware.auth_token [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 09:36:13.199013 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: WARNING keystonemiddleware.auth_token [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 09:36:13.542239 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: WARNING keystonemiddleware.auth_token [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 09:36:13.548434 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: WARNING keystonemiddleware.auth_token [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 09:36:13.548710 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: WARNING keystonemiddleware.auth_token [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 09:36:13.554494 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5563cc1d79c0 pid: 112051 (default app) Oct 28 09:36:13.555209 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: WARNING keystonemiddleware.auth_token [None req-a296fdfc-8c79-483f-9d0d-e00dc592ad1d None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 09:36:13.556846 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:36:13.558087 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] 10.222.0.25 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001901 Oct 28 09:36:13.559312 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 788 bytes} [Fri Oct 28 09:36:13 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:36:13.560786 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5563cc1d79c0 pid: 112050 (default app) Oct 28 09:37:36.905803 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:37:36.906856 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001498 Oct 28 09:37:36.907339 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 2/2] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:37:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:37:36.919520 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG dbcounter [-] [112050] Writing DB stats nova_api:SELECT=3 {{(pid=112050) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:36.920812 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG dbcounter [-] [112050] Writing DB stats nova_cell0:SELECT=3 {{(pid=112050) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:36.921666 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG dbcounter [-] [112050] Writing DB stats nova_cell1:SELECT=3 {{(pid=112050) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:36.922653 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG dbcounter [-] [112050] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=112050) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:37.809669 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-a0ad8543-eb74-4197-a456-4aebc4a7e5d9 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:37:37.842070 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-a0ad8543-eb74-4197-a456-4aebc4a7e5d9 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.927595 Oct 28 09:37:37.842840 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 1/3] 10.222.0.25 () {60 vars in 1292 bytes} [Fri Oct 28 09:37:36 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 930 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:37:40.522610 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:37:40.523496 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-85344c78-230a-40fe-86d8-21320a17b4a3 None None] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001606 Oct 28 09:37:40.524148 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 3/4] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:37:40 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:37:41.012389 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-5104d75b-ffe1-472f-8454-c006c8760060 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "c1", "rxtx_factor": 1.0, "name": "cirros256", "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "swap": 0, "ram": 256, "disk": 1, "vcpus": 1}} {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:37:41.060684 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-5104d75b-ffe1-472f-8454-c006c8760060 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.529000 Oct 28 09:37:41.061261 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 2/5] 10.222.0.25 () {64 vars in 1283 bytes} [Fri Oct 28 09:37:40 2022] POST /compute/v2.1/flavors => generated 407 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:37:41.075701 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writing DB stats nova_api:SELECT=3 {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:41.076565 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writing DB stats nova_cell1:SELECT=3 {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:41.078515 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writing DB stats nova_cell0:SELECT=3 {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:41.080238 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:41.285358 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-4f528035-249b-462a-87ae-c820bc38185a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:37:41.355054 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-4f528035-249b-462a-87ae-c820bc38185a admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.284806 Oct 28 09:37:41.355649 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 4/6] 10.222.0.25 () {64 vars in 1336 bytes} [Fri Oct 28 09:37:41 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:37:44.131459 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-5104d75b-ffe1-472f-8454-c006c8760060 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:37:44.132870 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-5104d75b-ffe1-472f-8454-c006c8760060 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002037 Oct 28 09:37:44.133477 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 3/7] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:37:44 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:37:44.820670 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-ff0e914e-9984-4dde-bea8-07643db68341 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds512M", "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "ram": 512, "disk": 5, "id": "d1", "swap": 0, "rxtx_factor": 1.0, "vcpus": 1}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:37:44.868383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-ff0e914e-9984-4dde-bea8-07643db68341 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.728343 Oct 28 09:37:44.870204 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 5/8] 10.222.0.25 () {64 vars in 1283 bytes} [Fri Oct 28 09:37:44 2022] POST /compute/v2.1/flavors => generated 404 bytes in 730 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:37:44.884354 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-9291373c-49ce-41ef-b2b2-40e232faeb89 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:37:44.927707 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-9291373c-49ce-41ef-b2b2-40e232faeb89 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.048244 Oct 28 09:37:44.938528 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 4/9] 10.222.0.25 () {64 vars in 1336 bytes} [Fri Oct 28 09:37:44 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:37:47.553620 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-ff0e914e-9984-4dde-bea8-07643db68341 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:37:47.554692 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-ff0e914e-9984-4dde-bea8-07643db68341 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001528 Oct 28 09:37:47.555109 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 6/10] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:37:47 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:37:47.987574 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-1701c458-afaf-42f0-8d02-4347488b9b60 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "ram": 1024, "rxtx_factor": 1.0, "id": "d2", "swap": 0, "name": "ds1G", "disk": 10, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true}} {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:37:48.007288 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-1701c458-afaf-42f0-8d02-4347488b9b60 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.445294 Oct 28 09:37:48.008183 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 5/11] 10.222.0.25 () {64 vars in 1283 bytes} [Fri Oct 28 09:37:47 2022] POST /compute/v2.1/flavors => generated 404 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:37:48.020554 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-9a6c2e24-a7b0-4399-b2cf-997eb7c6b505 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:37:48.050805 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-9a6c2e24-a7b0-4399-b2cf-997eb7c6b505 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.034576 Oct 28 09:37:48.051430 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 7/12] 10.222.0.25 () {64 vars in 1336 bytes} [Fri Oct 28 09:37:48 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:37:50.674929 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-1701c458-afaf-42f0-8d02-4347488b9b60 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:37:50.675933 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-1701c458-afaf-42f0-8d02-4347488b9b60 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001486 Oct 28 09:37:50.676339 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 6/13] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:37:50 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:37:51.161519 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-fbeba392-f535-4262-9fa2-700e1890aec8 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d3", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 2, "name": "ds2G", "disk": 10, "ram": 2048, "os-flavor-access:is_public": true, "swap": 0, "rxtx_factor": 1.0}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:37:51.185869 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-fbeba392-f535-4262-9fa2-700e1890aec8 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.502654 Oct 28 09:37:51.186577 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 8/14] 10.222.0.25 () {64 vars in 1283 bytes} [Fri Oct 28 09:37:50 2022] POST /compute/v2.1/flavors => generated 404 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:37:51.204362 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-376ce63d-9e29-4755-91bd-62706c860b45 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:37:51.235578 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-376ce63d-9e29-4755-91bd-62706c860b45 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035953 Oct 28 09:37:51.236180 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 7/15] 10.222.0.25 () {64 vars in 1336 bytes} [Fri Oct 28 09:37:51 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:37:53.891894 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-fbeba392-f535-4262-9fa2-700e1890aec8 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:37:53.892990 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-fbeba392-f535-4262-9fa2-700e1890aec8 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001982 Oct 28 09:37:53.893361 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 9/16] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:37:53 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:37:54.366367 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-8441aa48-fc87-40cd-a28b-a34b8bfc9cb2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 4, "disk": 20, "os-flavor-access:is_public": true, "id": "d4", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "ram": 4096, "rxtx_factor": 1.0, "name": "ds4G"}} {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:37:54.387955 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-8441aa48-fc87-40cd-a28b-a34b8bfc9cb2 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.487709 Oct 28 09:37:54.388539 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 8/17] 10.222.0.25 () {64 vars in 1283 bytes} [Fri Oct 28 09:37:53 2022] POST /compute/v2.1/flavors => generated 404 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:37:54.401700 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-004014ed-8209-429d-bef1-3287261f6eda admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:37:54.428217 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-004014ed-8209-429d-bef1-3287261f6eda admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030275 Oct 28 09:37:54.428813 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 10/18] 10.222.0.25 () {64 vars in 1336 bytes} [Fri Oct 28 09:37:54 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:37:57.337869 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-8441aa48-fc87-40cd-a28b-a34b8bfc9cb2 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:37:57.339319 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-8441aa48-fc87-40cd-a28b-a34b8bfc9cb2 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002037 Oct 28 09:37:57.339911 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 9/19] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:37:57 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:37:57.816285 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-a90d9b1c-fac2-4017-9921-4fd22bbec53c admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:37:57.852614 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-a90d9b1c-fac2-4017-9921-4fd22bbec53c admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.505524 Oct 28 09:37:57.853408 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 11/20] 10.222.0.25 () {60 vars in 1292 bytes} [Fri Oct 28 09:37:57 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:38:00.855070 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-8441aa48-fc87-40cd-a28b-a34b8bfc9cb2 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:00.856163 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-8441aa48-fc87-40cd-a28b-a34b8bfc9cb2 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001604 Oct 28 09:38:00.856609 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 10/21] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:38:00 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:38:01.297881 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-85abb557-4a19-4a9b-bb87-6caa0bed12df admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "rxtx_factor": 1.0, "disk": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "1", "name": "m1.tiny", "swap": 0, "os-flavor-access:is_public": true, "ram": 512}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:01.318460 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-85abb557-4a19-4a9b-bb87-6caa0bed12df admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.455215 Oct 28 09:38:01.319012 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 12/22] 10.222.0.25 () {64 vars in 1283 bytes} [Fri Oct 28 09:38:00 2022] POST /compute/v2.1/flavors => generated 402 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 28 09:38:01.331012 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-2925d9d0-5f98-4008-8413-ac6d4abd59d1 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:01.358378 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-2925d9d0-5f98-4008-8413-ac6d4abd59d1 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031002 Oct 28 09:38:01.359166 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 11/23] 10.222.0.25 () {64 vars in 1333 bytes} [Fri Oct 28 09:38:01 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:38:03.988262 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-85abb557-4a19-4a9b-bb87-6caa0bed12df admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:03.989302 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-85abb557-4a19-4a9b-bb87-6caa0bed12df admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001461 Oct 28 09:38:03.989741 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 13/24] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:38:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:38:04.387734 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-bfc28606-a65d-43f3-9922-18a146c352e5 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 20, "vcpus": 1, "id": "2", "rxtx_factor": 1.0, "ram": 2048, "name": "m1.small", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true}} {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:04.411932 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-bfc28606-a65d-43f3-9922-18a146c352e5 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.415993 Oct 28 09:38:04.412529 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 12/25] 10.222.0.25 () {64 vars in 1283 bytes} [Fri Oct 28 09:38:03 2022] POST /compute/v2.1/flavors => generated 405 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 28 09:38:04.425407 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-0d619282-9072-43e6-8e63-7d545f1d22be admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:04.452849 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-0d619282-9072-43e6-8e63-7d545f1d22be admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031538 Oct 28 09:38:04.454258 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 14/26] 10.222.0.25 () {64 vars in 1333 bytes} [Fri Oct 28 09:38:04 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:38:07.482683 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-bfc28606-a65d-43f3-9922-18a146c352e5 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:07.485132 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-bfc28606-a65d-43f3-9922-18a146c352e5 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002639 Oct 28 09:38:07.485942 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 13/27] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:38:07 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:38:08.161907 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-1ae07cba-f88c-4a34-a779-a59e195342b3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.medium", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 2, "disk": 40, "os-flavor-access:is_public": true, "id": "3", "rxtx_factor": 1.0, "swap": 0, "ram": 4096}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:08.194524 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-1ae07cba-f88c-4a34-a779-a59e195342b3 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.697212 Oct 28 09:38:08.195471 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 15/28] 10.222.0.25 () {64 vars in 1283 bytes} [Fri Oct 28 09:38:07 2022] POST /compute/v2.1/flavors => generated 406 bytes in 700 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:38:08.209820 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-4654807c-9336-4713-90c0-14c9c8b5f4f7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:08.215493 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG dbcounter [-] [112050] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=7,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=112050) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:08.247315 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-4654807c-9336-4713-90c0-14c9c8b5f4f7 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.041564 Oct 28 09:38:08.248346 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 14/29] 10.222.0.25 () {64 vars in 1333 bytes} [Fri Oct 28 09:38:08 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:38:11.643735 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-1ae07cba-f88c-4a34-a779-a59e195342b3 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:11.644940 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-1ae07cba-f88c-4a34-a779-a59e195342b3 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002190 Oct 28 09:38:11.645657 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 16/30] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:38:11 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:38:12.283701 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-94d7ec5a-ad04-4308-bee7-9df3233a25cf admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "name": "m1.large", "disk": 80, "swap": 0, "rxtx_factor": 1.0, "ram": 8192, "id": "4", "vcpus": 4, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:12.321596 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-94d7ec5a-ad04-4308-bee7-9df3233a25cf admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.668728 Oct 28 09:38:12.331698 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 15/31] 10.222.0.25 () {64 vars in 1283 bytes} [Fri Oct 28 09:38:11 2022] POST /compute/v2.1/flavors => generated 405 bytes in 679 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:38:12.343176 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-4b44bad8-439d-4397-8d90-4e41e0ec1186 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:12.349074 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writing DB stats nova_api:SELECT=39,nova_api:SAVEPOINT=4,nova_api:INSERT=8,nova_api:RELEASE=4 {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:12.374024 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-4b44bad8-439d-4397-8d90-4e41e0ec1186 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.040580 Oct 28 09:38:12.386640 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 17/32] 10.222.0.25 () {64 vars in 1333 bytes} [Fri Oct 28 09:38:12 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:38:15.809396 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-94d7ec5a-ad04-4308-bee7-9df3233a25cf admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:15.811381 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-94d7ec5a-ad04-4308-bee7-9df3233a25cf admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002457 Oct 28 09:38:15.811892 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 16/33] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:38:15 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:38:16.404359 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-50676d78-83e9-4818-9050-327839614c88 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 8, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 160, "ram": 16384, "swap": 0, "name": "m1.xlarge", "os-flavor-access:is_public": true, "id": "5", "rxtx_factor": 1.0}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:16.441738 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-50676d78-83e9-4818-9050-327839614c88 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.619818 Oct 28 09:38:16.441738 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 18/34] 10.222.0.25 () {64 vars in 1283 bytes} [Fri Oct 28 09:38:15 2022] POST /compute/v2.1/flavors => generated 408 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:38:16.451840 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-c6d9264d-d6e9-4757-9dec-cc2ee4ab7982 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:16.483020 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-c6d9264d-d6e9-4757-9dec-cc2ee4ab7982 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.034602 Oct 28 09:38:16.483967 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 17/35] 10.222.0.25 () {64 vars in 1333 bytes} [Fri Oct 28 09:38:16 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:38:32.886377 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-50676d78-83e9-4818-9050-327839614c88 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:32.887423 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-50676d78-83e9-4818-9050-327839614c88 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001497 Oct 28 09:38:32.887832 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 19/36] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:38:32 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:38:32.899216 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG dbcounter [-] [112050] Writing DB stats nova_api:SELECT=14,nova_api:SAVEPOINT=2,nova_api:INSERT=3,nova_api:RELEASE=2 {{(pid=112050) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:33.374196 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-15e09832-ede5-44b2-9892-ee1ae9b1a067 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:33.395071 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-15e09832-ede5-44b2-9892-ee1ae9b1a067 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.500264 Oct 28 09:38:33.395643 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 18/37] 10.222.0.25 () {60 vars in 1292 bytes} [Fri Oct 28 09:38:32 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:38:34.980427 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-50676d78-83e9-4818-9050-327839614c88 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:34.982076 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-50676d78-83e9-4818-9050-327839614c88 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001999 Oct 28 09:38:34.982782 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 20/38] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Oct 28 09:38:34 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:38:35.575515 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-31e41c26-84df-4b75-95c7-063b3009efd4 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:35.582149 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.wsgi [None req-31e41c26-84df-4b75-95c7-063b3009efd4 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-31e41c26-84df-4b75-95c7-063b3009efd4 admin admin] Request method failure captured: Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Accept: application/json Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Accept-Encoding: gzip, deflate Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Connection: keep-alive Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Content-Length: 0 Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Host: 10.222.0.25 Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Openstack-Api-Version: compute 2.92 Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Openstack-System-Scope: None Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: User-Agent: python-novaclient Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Auth-Token: gAAAAABjW6MajFMXSGB1qHteJ5JJgmnds-gGI_X-HPGHlMmvys4A7eo75hAf0frMXW5jtoWdBHcXGF4KnOu9fqLKXzKFpL9HvHrS72WjvDhPrTOS9oQXx1i20Qg0U7V08tD6o5Dgl8LuzuMig11crA9xR55QdCKGvSioruPheJSvkF6-9l3i5gI Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Domain-Id: None Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Domain-Name: None Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Identity-Status: Confirmed Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Is-Admin-Project: True Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Openstack-Nova-Api-Version: 2.92 Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Project-Domain-Id: default Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Project-Domain-Name: Default Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Project-Id: ac54d153685c4b2288a7ed215d0e24d2 Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Project-Name: admin Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Role: member,admin,reader Oct 28 09:38:35.585420 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Roles: member,admin,reader Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ac54d153685c4b2288a7ed215d0e24d2"}]}] Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Tenant: admin Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Tenant-Id: ac54d153685c4b2288a7ed215d0e24d2 Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Tenant-Name: admin Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User: admin Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User-Domain-Id: default Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User-Domain-Name: Default Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User-Id: 8afbf6a033494d4cbbdea7a924ab5c63 Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User-Name: admin Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: method: > Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: exception: Flavor 42 could not be found. Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: action_args: {'id': '42'} Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:38:35.586930 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:38:35.589541 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:35.591010 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-31e41c26-84df-4b75-95c7-063b3009efd4 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=112050) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:38:35.592234 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-31e41c26-84df-4b75-95c7-063b3009efd4 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.458740 Oct 28 09:38:35.592234 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 19/39] 10.222.0.25 () {60 vars in 1190 bytes} [Fri Oct 28 09:38:35 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 460 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Oct 28 09:38:35.597607 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-1e9d96f2-9e3e-4de8-a735-c0566d5e7d18 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:35.602210 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writing DB stats nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:INSERT=2,nova_api:RELEASE=1 {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:35.625062 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-1e9d96f2-9e3e-4de8-a735-c0566d5e7d18 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.030595 Oct 28 09:38:35.625848 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 21/40] 10.222.0.25 () {60 vars in 1210 bytes} [Fri Oct 28 09:38:35 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:38:35.637755 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-291174a0-3793-498a-a3fa-76b5ddd2ad62 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:35.666693 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-291174a0-3793-498a-a3fa-76b5ddd2ad62 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.034052 Oct 28 09:38:35.667318 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 20/41] 10.222.0.25 () {60 vars in 1210 bytes} [Fri Oct 28 09:38:35 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:38:35.678065 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-6df5dd0e-047b-4258-bffd-4f9ee283b57c admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:35.684032 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.wsgi [None req-6df5dd0e-047b-4258-bffd-4f9ee283b57c admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-6df5dd0e-047b-4258-bffd-4f9ee283b57c admin admin] Request method failure captured: Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Accept: application/json Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Accept-Encoding: gzip, deflate Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Connection: keep-alive Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Content-Length: 0 Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Host: 10.222.0.25 Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Openstack-Api-Version: compute 2.92 Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Openstack-System-Scope: None Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: User-Agent: python-novaclient Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Auth-Token: gAAAAABjW6MajFMXSGB1qHteJ5JJgmnds-gGI_X-HPGHlMmvys4A7eo75hAf0frMXW5jtoWdBHcXGF4KnOu9fqLKXzKFpL9HvHrS72WjvDhPrTOS9oQXx1i20Qg0U7V08tD6o5Dgl8LuzuMig11crA9xR55QdCKGvSioruPheJSvkF6-9l3i5gI Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Domain-Id: None Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Domain-Name: None Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Identity-Status: Confirmed Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Is-Admin-Project: True Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Openstack-Nova-Api-Version: 2.92 Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Project-Domain-Id: default Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Project-Domain-Name: Default Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Project-Id: ac54d153685c4b2288a7ed215d0e24d2 Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Project-Name: admin Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Role: member,admin,reader Oct 28 09:38:35.688026 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Roles: member,admin,reader Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ac54d153685c4b2288a7ed215d0e24d2"}]}] Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Tenant: admin Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Tenant-Id: ac54d153685c4b2288a7ed215d0e24d2 Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Tenant-Name: admin Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User: admin Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User-Domain-Id: default Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User-Domain-Name: Default Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User-Id: 8afbf6a033494d4cbbdea7a924ab5c63 Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User-Name: admin Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: method: > Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: exception: Flavor 42 could not be found. Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: action_args: {'id': '42'} Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:38:35.689770 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:38:35.690844 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:35.692533 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-6df5dd0e-047b-4258-bffd-4f9ee283b57c admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=112051) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:38:35.693856 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-6df5dd0e-047b-4258-bffd-4f9ee283b57c admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.015848 Oct 28 09:38:35.693856 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 22/42] 10.222.0.25 () {60 vars in 1190 bytes} [Fri Oct 28 09:38:35 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:36.997182 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-291174a0-3793-498a-a3fa-76b5ddd2ad62 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:36.998192 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-291174a0-3793-498a-a3fa-76b5ddd2ad62 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001499 Oct 28 09:38:36.998627 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 21/43] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Oct 28 09:38:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:38:37.539121 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-9f864cab-eea9-4932-b954-ee71f9b2fea4 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:37.544163 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.wsgi [None req-9f864cab-eea9-4932-b954-ee71f9b2fea4 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-9f864cab-eea9-4932-b954-ee71f9b2fea4 admin admin] Request method failure captured: Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Accept: application/json Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Accept-Encoding: gzip, deflate Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Connection: keep-alive Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Content-Length: 0 Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Host: 10.222.0.25 Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Openstack-Api-Version: compute 2.92 Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Openstack-System-Scope: None Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: User-Agent: python-novaclient Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Auth-Token: gAAAAABjW6McjKYeG736IVOVEpTjVamg_oihHGg1m_Mt7al86h7-JD8cMEYjM5Vit9GBvARYtRbSIqjaElo9IrOeVFpPAbm-8ysjidPHrHZEC-dVIqRiRUJSG-5In2lj7PocMjBIW2bzbNNWonLuzZf90vwzO9BHfBLxAuCPf2ivbWndUEnXPhY Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Domain-Id: None Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Domain-Name: None Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Identity-Status: Confirmed Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Is-Admin-Project: True Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Openstack-Nova-Api-Version: 2.92 Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Project-Domain-Id: default Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Project-Domain-Name: Default Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Project-Id: ac54d153685c4b2288a7ed215d0e24d2 Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Project-Name: admin Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Role: member,admin,reader Oct 28 09:38:37.545651 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Roles: member,admin,reader Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ac54d153685c4b2288a7ed215d0e24d2"}]}] Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Tenant: admin Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Tenant-Id: ac54d153685c4b2288a7ed215d0e24d2 Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Tenant-Name: admin Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User: admin Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User-Domain-Id: default Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User-Domain-Name: Default Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User-Id: 8afbf6a033494d4cbbdea7a924ab5c63 Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User-Name: admin Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: method: > Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: exception: Flavor 84 could not be found. Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: action_args: {'id': '84'} Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:38:37.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:38:37.548383 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:37.549749 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-9f864cab-eea9-4932-b954-ee71f9b2fea4 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=112051) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:38:37.551113 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-9f864cab-eea9-4932-b954-ee71f9b2fea4 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.410898 Oct 28 09:38:37.551113 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 23/44] 10.222.0.25 () {60 vars in 1190 bytes} [Fri Oct 28 09:38:37 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 412 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:37.556961 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-3baaa7d3-f810-4c9c-aba6-a3a77b46c252 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:37.572000 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-3baaa7d3-f810-4c9c-aba6-a3a77b46c252 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018616 Oct 28 09:38:37.572556 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 22/45] 10.222.0.25 () {60 vars in 1210 bytes} [Fri Oct 28 09:38:37 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:38:37.582754 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-7ccf80df-9c73-4d7b-af72-a5615dda4859 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:37.598883 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-7ccf80df-9c73-4d7b-af72-a5615dda4859 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019678 Oct 28 09:38:37.599416 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 24/46] 10.222.0.25 () {60 vars in 1210 bytes} [Fri Oct 28 09:38:37 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:38:37.608803 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-3a28d6f3-978c-40bc-b115-d7b516019358 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:37.613847 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.wsgi [None req-3a28d6f3-978c-40bc-b115-d7b516019358 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-3a28d6f3-978c-40bc-b115-d7b516019358 admin admin] Request method failure captured: Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Accept: application/json Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Accept-Encoding: gzip, deflate Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Connection: keep-alive Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Content-Length: 0 Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Host: 10.222.0.25 Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Openstack-Api-Version: compute 2.92 Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Openstack-System-Scope: None Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: User-Agent: python-novaclient Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Auth-Token: gAAAAABjW6McjKYeG736IVOVEpTjVamg_oihHGg1m_Mt7al86h7-JD8cMEYjM5Vit9GBvARYtRbSIqjaElo9IrOeVFpPAbm-8ysjidPHrHZEC-dVIqRiRUJSG-5In2lj7PocMjBIW2bzbNNWonLuzZf90vwzO9BHfBLxAuCPf2ivbWndUEnXPhY Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Domain-Id: None Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Domain-Name: None Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Identity-Status: Confirmed Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Is-Admin-Project: True Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Openstack-Nova-Api-Version: 2.92 Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Project-Domain-Id: default Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Project-Domain-Name: Default Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Project-Id: ac54d153685c4b2288a7ed215d0e24d2 Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Project-Name: admin Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Role: member,admin,reader Oct 28 09:38:37.615399 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Roles: member,admin,reader Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ac54d153685c4b2288a7ed215d0e24d2"}]}] Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Tenant: admin Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Tenant-Id: ac54d153685c4b2288a7ed215d0e24d2 Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Tenant-Name: admin Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User: admin Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User-Domain-Id: default Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User-Domain-Name: Default Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User-Id: 8afbf6a033494d4cbbdea7a924ab5c63 Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User-Name: admin Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: method: > Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: exception: Flavor 84 could not be found. Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: action_args: {'id': '84'} Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:38:37.616728 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:38:37.617511 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:37.618757 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-3a28d6f3-978c-40bc-b115-d7b516019358 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=112050) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:38:37.619931 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-3a28d6f3-978c-40bc-b115-d7b516019358 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011603 Oct 28 09:38:37.619931 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 23/47] 10.222.0.25 () {60 vars in 1190 bytes} [Fri Oct 28 09:38:37 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Oct 28 09:38:38.871838 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-7ccf80df-9c73-4d7b-af72-a5615dda4859 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:38.872849 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-7ccf80df-9c73-4d7b-af72-a5615dda4859 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001443 Oct 28 09:38:38.873254 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 25/48] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Oct 28 09:38:38 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:38:39.417304 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-4b3b4544-ba39-4860-ad57-cb04d84ccfb2 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:39.423870 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.wsgi [None req-4b3b4544-ba39-4860-ad57-cb04d84ccfb2 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-4b3b4544-ba39-4860-ad57-cb04d84ccfb2 admin admin] Request method failure captured: Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Accept: application/json Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Accept-Encoding: gzip, deflate Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Connection: keep-alive Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Content-Length: 0 Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Host: 10.222.0.25 Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Openstack-Api-Version: compute 2.92 Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Openstack-System-Scope: None Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: User-Agent: python-novaclient Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Auth-Token: gAAAAABjW6MelwZfHNCS-U8l1N4fHiSrCpa6IhhtVg3gd15LWqJI0wJeVCUBCw_PbZtK7bI2Y0jwdnoT2DVmmILqg91gTJ4eAGxvhSPZ3Jql7EY3Ek6dovhoINjEld2T3azZkCNuw0EQUeknH5j4jntPr1g8My_Y8HN0x9ZikJ2bh6ewgphf8ZI Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Domain-Id: None Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Domain-Name: None Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Identity-Status: Confirmed Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Is-Admin-Project: True Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Openstack-Nova-Api-Version: 2.92 Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Project-Domain-Id: default Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Project-Domain-Name: Default Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Project-Id: ac54d153685c4b2288a7ed215d0e24d2 Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Project-Name: admin Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Role: member,admin,reader Oct 28 09:38:39.425393 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Roles: member,admin,reader Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ac54d153685c4b2288a7ed215d0e24d2"}]}] Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Tenant: admin Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Tenant-Id: ac54d153685c4b2288a7ed215d0e24d2 Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-Tenant-Name: admin Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User: admin Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User-Domain-Id: default Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User-Domain-Name: Default Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User-Id: 8afbf6a033494d4cbbdea7a924ab5c63 Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: X-User-Name: admin Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: method: > Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: exception: Flavor 451 could not be found. Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: action_args: {'id': '451'} Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:38:39.428034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:38:39.429598 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: ERROR nova.api.openstack.wsgi Oct 28 09:38:39.431954 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-4b3b4544-ba39-4860-ad57-cb04d84ccfb2 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=112050) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:38:39.434215 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-4b3b4544-ba39-4860-ad57-cb04d84ccfb2 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.416138 Oct 28 09:38:39.434215 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 24/49] 10.222.0.25 () {60 vars in 1193 bytes} [Fri Oct 28 09:38:39 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 417 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:39.436881 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-a06700f5-0ad6-47d8-842d-23716178730b admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:39.453020 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-a06700f5-0ad6-47d8-842d-23716178730b admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019284 Oct 28 09:38:39.453580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 26/50] 10.222.0.25 () {60 vars in 1210 bytes} [Fri Oct 28 09:38:39 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:38:39.463367 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:39.478939 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019049 Oct 28 09:38:39.479490 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 25/51] 10.222.0.25 () {60 vars in 1210 bytes} [Fri Oct 28 09:38:39 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:38:39.489149 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-8203c671-06b8-46a9-87d8-573c75bb3f01 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:39.494230 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.wsgi [None req-8203c671-06b8-46a9-87d8-573c75bb3f01 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-8203c671-06b8-46a9-87d8-573c75bb3f01 admin admin] Request method failure captured: Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Accept: application/json Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Accept-Encoding: gzip, deflate Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Connection: keep-alive Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Content-Length: 0 Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Host: 10.222.0.25 Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Openstack-Api-Version: compute 2.92 Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Openstack-System-Scope: None Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: User-Agent: python-novaclient Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Auth-Token: gAAAAABjW6MelwZfHNCS-U8l1N4fHiSrCpa6IhhtVg3gd15LWqJI0wJeVCUBCw_PbZtK7bI2Y0jwdnoT2DVmmILqg91gTJ4eAGxvhSPZ3Jql7EY3Ek6dovhoINjEld2T3azZkCNuw0EQUeknH5j4jntPr1g8My_Y8HN0x9ZikJ2bh6ewgphf8ZI Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Domain-Id: None Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Domain-Name: None Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Identity-Status: Confirmed Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Is-Admin-Project: True Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Openstack-Nova-Api-Version: 2.92 Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Project-Domain-Id: default Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Project-Domain-Name: Default Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Project-Id: ac54d153685c4b2288a7ed215d0e24d2 Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Project-Name: admin Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Role: member,admin,reader Oct 28 09:38:39.495890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Roles: member,admin,reader Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ac54d153685c4b2288a7ed215d0e24d2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ac54d153685c4b2288a7ed215d0e24d2"}]}] Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Tenant: admin Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Tenant-Id: ac54d153685c4b2288a7ed215d0e24d2 Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-Tenant-Name: admin Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User: admin Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User-Domain-Id: default Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User-Domain-Name: Default Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User-Id: 8afbf6a033494d4cbbdea7a924ab5c63 Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: X-User-Name: admin Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: method: > Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: exception: Flavor 451 could not be found. Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: action_args: {'id': '451'} Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:38:39.497228 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:38:39.498035 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: ERROR nova.api.openstack.wsgi Oct 28 09:38:39.499580 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-8203c671-06b8-46a9-87d8-573c75bb3f01 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=112051) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:38:39.500950 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-8203c671-06b8-46a9-87d8-573c75bb3f01 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011830 Oct 28 09:38:39.500950 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 27/52] 10.222.0.25 () {60 vars in 1193 bytes} [Fri Oct 28 09:38:39 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:40.734482 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:40.735493 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002035 Oct 28 09:38:40.735892 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 26/53] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Oct 28 09:38:40 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:38:41.267923 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-e9f3ba0d-99ef-419b-bf9a-d9b6b98332e3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:41.289873 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-e9f3ba0d-99ef-419b-bf9a-d9b6b98332e3 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.414463 Oct 28 09:38:41.290454 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 28/54] 10.222.0.25 () {64 vars in 1235 bytes} [Fri Oct 28 09:38:40 2022] POST /compute/v2.1/flavors => generated 404 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:38:42.515487 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:42.516508 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001464 Oct 28 09:38:42.516921 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 27/55] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Oct 28 09:38:42 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:38:43.037176 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-3daa531b-ad55-4efd-96c7-2c7ecaf9a3c9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:43.056330 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-3daa531b-ad55-4efd-96c7-2c7ecaf9a3c9 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.401789 Oct 28 09:38:43.056907 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 29/56] 10.222.0.25 () {64 vars in 1235 bytes} [Fri Oct 28 09:38:42 2022] POST /compute/v2.1/flavors => generated 405 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:38:44.308177 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:44.309197 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001458 Oct 28 09:38:44.309603 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 28/57] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Oct 28 09:38:44 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:38:44.840516 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-7327d395-fc18-42db-bffd-d7c91eba9b01 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:44.862005 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-7327d395-fc18-42db-bffd-d7c91eba9b01 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.413949 Oct 28 09:38:44.862591 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 30/58] 10.222.0.25 () {64 vars in 1235 bytes} [Fri Oct 28 09:38:44 2022] POST /compute/v2.1/flavors => generated 407 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:38:46.087045 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:46.088075 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001487 Oct 28 09:38:46.088471 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 29/59] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Oct 28 09:38:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:38:46.609066 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-8036e1d1-9965-4305-91c6-ace0841e6156 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:38:46.627368 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-8036e1d1-9965-4305-91c6-ace0841e6156 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.403253 Oct 28 09:38:46.627930 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 31/60] 10.222.0.25 () {64 vars in 1235 bytes} [Fri Oct 28 09:38:46 2022] POST /compute/v2.1/flavors => generated 409 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 28 09:38:47.862416 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:47.863437 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001526 Oct 28 09:38:47.863844 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 30/61] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Oct 28 09:38:47 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:38:48.429148 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-bd40cd01-3ab3-4db2-ac9b-171801bfc1e6 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:38:48.449776 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-bd40cd01-3ab3-4db2-ac9b-171801bfc1e6 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.447655 Oct 28 09:38:48.450317 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 32/62] 10.222.0.25 () {60 vars in 1202 bytes} [Fri Oct 28 09:38:48 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:39:41.187798 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:39:41.188960 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-e9e128d1-037d-45b8-aa1e-c999587fc376 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001778 Oct 28 09:39:41.189450 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 31/63] 10.222.0.25 () {54 vars in 931 bytes} [Fri Oct 28 09:39:41 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:39:41.200757 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:41.588957 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-540a2a42-5a32-4a40-b831-251db799ca39 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:39:41.610890 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-540a2a42-5a32-4a40-b831-251db799ca39 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.414604 Oct 28 09:39:41.611558 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 33/64] 10.222.0.25 () {60 vars in 1292 bytes} [Fri Oct 28 09:39:41 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:39:41.632564 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG dbcounter [-] [112050] Writing DB stats nova_api:SELECT=14 {{(pid=112050) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:41.636416 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-e46136bd-c1a9-4c6b-992c-c2410e639a57 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:39:41.644353 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-e46136bd-c1a9-4c6b-992c-c2410e639a57 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013232 Oct 28 09:39:41.644890 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 32/65] 10.222.0.25 () {60 vars in 1283 bytes} [Fri Oct 28 09:39:41 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:39:41.657385 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-a68af57f-2e54-4eec-bd87-c4fdd7f3eb3c admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:39:41.664926 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-a68af57f-2e54-4eec-bd87-c4fdd7f3eb3c admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011750 Oct 28 09:39:41.665442 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 34/66] 10.222.0.25 () {60 vars in 1286 bytes} [Fri Oct 28 09:39:41 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:39:41.677034 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-a5680ce9-abce-4f58-bf46-3e66202e91c6 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:39:41.684264 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-a5680ce9-abce-4f58-bf46-3e66202e91c6 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011027 Oct 28 09:39:41.684786 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 33/67] 10.222.0.25 () {60 vars in 1286 bytes} [Fri Oct 28 09:39:41 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:39:41.696355 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-26efa1fc-1136-4253-b274-f09b873295da admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:39:41.704118 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-26efa1fc-1136-4253-b274-f09b873295da admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011374 Oct 28 09:39:41.704640 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 35/68] 10.222.0.25 () {60 vars in 1283 bytes} [Fri Oct 28 09:39:41 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:42:54.209102 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-a5680ce9-abce-4f58-bf46-3e66202e91c6 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:42:54.210120 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-a5680ce9-abce-4f58-bf46-3e66202e91c6 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001456 Oct 28 09:42:54.210554 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 34/69] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Oct 28 09:42:54 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:42:54.350506 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writing DB stats nova_api:SELECT=6 {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:42:54.513027 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-6b4171a7-eef3-48ca-b900-2a083363affb admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:42:54.517872 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-6b4171a7-eef3-48ca-b900-2a083363affb admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:42:54.518344 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-6b4171a7-eef3-48ca-b900-2a083363affb admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:42:54.518798 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-6b4171a7-eef3-48ca-b900-2a083363affb admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:42:54.519504 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-6b4171a7-eef3-48ca-b900-2a083363affb admin admin] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:42:54.519944 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-6b4171a7-eef3-48ca-b900-2a083363affb admin admin] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:42:54.520358 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-6b4171a7-eef3-48ca-b900-2a083363affb admin admin] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:42:54.552203 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-6b4171a7-eef3-48ca-b900-2a083363affb admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 782 microversion: 2.92 time: 0.202923 Oct 28 09:42:54.552768 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 36/70] 10.222.0.25 () {60 vars in 1193 bytes} [Fri Oct 28 09:42:54 2022] GET /compute/v2.1/os-services => generated 782 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:42:55.648860 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-a5680ce9-abce-4f58-bf46-3e66202e91c6 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:42:55.649928 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-a5680ce9-abce-4f58-bf46-3e66202e91c6 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Oct 28 09:42:55.650334 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 35/71] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Oct 28 09:42:55 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:42:55.840826 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-3e108f11-1d3f-4ddb-a8f7-746f23c5a815 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:42:55.844488 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-3e108f11-1d3f-4ddb-a8f7-746f23c5a815 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:42:55.844973 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-3e108f11-1d3f-4ddb-a8f7-746f23c5a815 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:42:55.845405 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-3e108f11-1d3f-4ddb-a8f7-746f23c5a815 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:42:55.846136 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-3e108f11-1d3f-4ddb-a8f7-746f23c5a815 admin admin] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:42:55.846650 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-3e108f11-1d3f-4ddb-a8f7-746f23c5a815 admin admin] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:42:55.847070 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-3e108f11-1d3f-4ddb-a8f7-746f23c5a815 admin admin] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:42:55.867025 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-3e108f11-1d3f-4ddb-a8f7-746f23c5a815 admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 782 microversion: 2.92 time: 0.079091 Oct 28 09:42:55.867613 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 37/72] 10.222.0.25 () {60 vars in 1193 bytes} [Fri Oct 28 09:42:55 2022] GET /compute/v2.1/os-services => generated 782 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 28 09:43:07.382094 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-a5680ce9-abce-4f58-bf46-3e66202e91c6 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:43:07.383140 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-a5680ce9-abce-4f58-bf46-3e66202e91c6 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 Oct 28 09:43:07.383574 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 36/73] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Oct 28 09:43:07 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:43:07.528865 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writing DB stats nova_cell0:SELECT=4 {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:43:07.529584 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writing DB stats nova_cell1:SELECT=4 {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:43:07.530198 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG dbcounter [-] [112051] Writing DB stats nova_api:SELECT=8 {{(pid=112051) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:43:07.582246 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-88da29b5-8532-44e8-8d48-f969075b75c9 admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:43:07.585680 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-88da29b5-8532-44e8-8d48-f969075b75c9 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:43:07.586171 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-88da29b5-8532-44e8-8d48-f969075b75c9 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:43:07.587143 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-88da29b5-8532-44e8-8d48-f969075b75c9 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:43:07.587982 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-88da29b5-8532-44e8-8d48-f969075b75c9 admin admin] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:43:07.588376 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-88da29b5-8532-44e8-8d48-f969075b75c9 admin admin] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:43:07.588787 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-88da29b5-8532-44e8-8d48-f969075b75c9 admin admin] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:43:07.614344 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-88da29b5-8532-44e8-8d48-f969075b75c9 admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 1002 microversion: 2.92 time: 0.090854 Oct 28 09:43:07.614962 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 38/74] 10.222.0.25 () {60 vars in 1193 bytes} [Fri Oct 28 09:43:07 2022] GET /compute/v2.1/os-services => generated 1002 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:43:08.707202 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG nova.api.openstack.wsgi [None req-a5680ce9-abce-4f58-bf46-3e66202e91c6 admin admin] Calling method '>' {{(pid=112050) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:43:08.708211 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: INFO nova.api.openstack.requestlog [None req-a5680ce9-abce-4f58-bf46-3e66202e91c6 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001474 Oct 28 09:43:08.708605 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: [pid: 112050|app: 0|req: 37/75] 10.222.0.25 () {56 vars in 1074 bytes} [Fri Oct 28 09:43:08 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:43:08.897119 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG nova.api.openstack.wsgi [None req-00cdbca4-6ba5-442d-8df7-91b7dc1dcd2c admin admin] Calling method '>' {{(pid=112051) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:43:08.901014 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-00cdbca4-6ba5-442d-8df7-91b7dc1dcd2c admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:43:08.901525 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-00cdbca4-6ba5-442d-8df7-91b7dc1dcd2c admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:43:08.901957 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-00cdbca4-6ba5-442d-8df7-91b7dc1dcd2c admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:43:08.902707 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-00cdbca4-6ba5-442d-8df7-91b7dc1dcd2c admin admin] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:43:08.903160 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-00cdbca4-6ba5-442d-8df7-91b7dc1dcd2c admin admin] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:43:08.903579 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: DEBUG oslo_concurrency.lockutils [None req-00cdbca4-6ba5-442d-8df7-91b7dc1dcd2c admin admin] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112051) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:43:08.925824 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: INFO nova.api.openstack.requestlog [None req-00cdbca4-6ba5-442d-8df7-91b7dc1dcd2c admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 1002 microversion: 2.92 time: 0.080734 Oct 28 09:43:08.926413 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: [pid: 112051|app: 0|req: 39/76] 10.222.0.25 () {60 vars in 1193 bytes} [Fri Oct 28 09:43:08 2022] GET /compute/v2.1/os-services => generated 1002 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:44:07.974854 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: Fri Oct 28 09:44:07 2022 - graceful shutdown triggered... Oct 28 09:44:07.975396 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: Gracefully killing worker 1 (pid: 112050)... Oct 28 09:44:07.975765 dv-cinder-zed-3-iscsi devstack@n-api.service[112051]: Gracefully killing worker 2 (pid: 112051)... Oct 28 09:44:07.977216 dv-cinder-zed-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Oct 28 09:44:08.100961 dv-cinder-zed-3-iscsi devstack@n-api.service[112050]: DEBUG dbcounter [-] [112050] Writing DB stats nova_api:SELECT=4 {{(pid=112050) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:44:08.976011 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: worker 1 buried after 1 seconds Oct 28 09:44:08.976011 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: worker 2 buried after 1 seconds Oct 28 09:44:08.977237 dv-cinder-zed-3-iscsi devstack@n-api.service[112049]: goodbye to uWSGI. Oct 28 09:44:08.978179 dv-cinder-zed-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Oct 28 09:44:08.978782 dv-cinder-zed-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Oct 28 09:44:08.981559 dv-cinder-zed-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 28 09:44:08.992799 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 28 09:44:08.995598 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Oct 28 09:44:08 2022] *** Oct 28 09:44:08.995598 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 28 09:44:08.995598 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 28 09:44:08.995598 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: nodename: dv-cinder-zed-3-iscsi Oct 28 09:44:08.995598 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: machine: x86_64 Oct 28 09:44:08.995598 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: clock source: unix Oct 28 09:44:08.995598 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: pcre jit disabled Oct 28 09:44:08.995598 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: detected number of CPU cores: 4 Oct 28 09:44:08.995598 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: current working directory: / Oct 28 09:44:08.995598 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: detected binary path: /usr/bin/uwsgi-core Oct 28 09:44:08.995598 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: your processes number limit is 47680 Oct 28 09:44:08.996317 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: your memory page size is 4096 bytes Oct 28 09:44:08.996317 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: detected max file descriptor number: 2048 Oct 28 09:44:08.996317 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: lock engine: pthread robust mutexes Oct 28 09:44:08.996317 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: thunder lock: enabled Oct 28 09:44:08.996317 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 28 09:44:08.996317 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 28 09:44:09.035242 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: Python main interpreter initialized at 0x55e864e319c0 Oct 28 09:44:09.035242 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: python threads support enabled Oct 28 09:44:09.035242 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: your server socket listen backlog is limited to 100 connections Oct 28 09:44:09.035242 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: your mercy for graceful operations on workers is 90 seconds Oct 28 09:44:09.035775 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: mapped 403077 bytes (393 KB) for 2 cores Oct 28 09:44:09.035775 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: *** Operational MODE: preforking *** Oct 28 09:44:09.035935 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: *** uWSGI is running in multiple interpreter mode *** Oct 28 09:44:09.035935 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: spawned uWSGI master process (pid: 127016) Oct 28 09:44:09.036136 dv-cinder-zed-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 28 09:44:09.036726 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: spawned uWSGI worker 1 (pid: 127029, cores: 1) Oct 28 09:44:09.037230 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: spawned uWSGI worker 2 (pid: 127030, cores: 1) Oct 28 09:44:09.037513 dv-cinder-zed-3-iscsi devstack@n-api.service[127016]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 28 09:44:09.710425 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 28 09:44:09.717518 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 28 09:44:10.873130 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 09:44:10.873130 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: from cryptography.utils import int_from_bytes Oct 28 09:44:10.874023 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 09:44:10.874023 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: from cryptography.utils import int_from_bytes Oct 28 09:44:10.878833 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 09:44:10.878833 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: from cryptography.utils import int_from_bytes Oct 28 09:44:10.879306 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 09:44:10.879306 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: from cryptography.utils import int_from_bytes Oct 28 09:44:13.324416 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 28 09:44:13.325045 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 28 09:44:13.325319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 28 09:44:13.325592 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 28 09:44:13.325862 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 28 09:44:13.326255 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.326614 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.326937 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.327257 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.327585 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.327901 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.328236 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.328556 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.328905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.329155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.329471 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.329969 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-zed-3-iscsi {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.330303 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.330651 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.330973 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.331302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.331612 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.331938 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.332257 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.332761 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.334209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.334209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.334209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.334209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.334810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.334810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.335095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.335448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.336312 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.336642 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.337037 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-zed-3-iscsi {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.337357 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.337662 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.337972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.338433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.338743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.339056 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.339370 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.339689 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.339995 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.340323 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.340646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.340961 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.341298 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.341589 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.341900 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.342209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.342556 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.342888 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.343172 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.343274 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 28 09:44:13.343681 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.343757 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 28 09:44:13.343833 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.344022 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 28 09:44:13.344115 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.344326 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 28 09:44:13.344458 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.344613 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 28 09:44:13.344716 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.344895 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.345003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.345151 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.345302 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.345466 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.345624 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.345776 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.345925 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.346073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.346223 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.346423 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.346598 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.346759 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.346910 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.347064 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.347205 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.347375 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.347527 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.347674 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.347814 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.348001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.348158 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.348313 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.348485 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-zed-3-iscsi {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.348627 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.348766 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.348911 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.349079 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.349266 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.349440 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.349584 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.349731 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.349907 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.350066 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.350214 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.350362 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.350590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.350740 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.350910 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.351155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.351395 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.351643 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.351805 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.351972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.352134 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.352284 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.352431 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.352592 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.352740 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.352887 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.353042 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.353209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.354187 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.354342 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.354581 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.354732 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.354902 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.355052 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.355193 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.355343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.355507 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.355665 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.355665 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.355947 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-zed-3-iscsi {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.356136 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.356299 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.356449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.356603 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.356755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.357016 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.357219 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.357376 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.357542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.357701 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.357853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.358001 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.358187 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.358332 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.358514 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.358667 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.358873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.359056 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.359208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.359355 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.359513 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.359672 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.359821 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.359821 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.360096 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.360242 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.360398 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.360547 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.360696 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.360843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.360983 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.360983 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.361311 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.361460 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.361627 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.361776 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.361916 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.362069 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.362218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.362611 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.362849 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.363037 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.363200 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.363342 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.363510 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.364440 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.364611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.364766 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.364909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.365056 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.365241 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.365405 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.365549 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.365695 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.365844 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.365993 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.366150 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.366299 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.366482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.366645 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.366788 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.366937 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.367084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.367233 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.367384 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.367534 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.367681 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.367832 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.367983 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.368134 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.368330 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.368497 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.368645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.368784 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.368938 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.369087 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.369227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.369227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.369504 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.369650 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.369803 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.369944 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.370094 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.370234 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.370383 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.370546 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.370698 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.370865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.371020 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.371167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.371321 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.371477 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.371625 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.371773 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.371919 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.372060 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.372218 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.372368 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.372522 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.372664 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.372818 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.372980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.373123 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.373276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.373438 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.373601 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.373773 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.374777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.374944 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.375118 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.375267 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.375414 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.375554 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.375843 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.375995 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.376138 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.376285 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.376432 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.376581 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.376730 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.376887 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.377028 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.377190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.377337 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.377483 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.377638 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.377786 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.377933 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.378073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.378214 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.378214 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.378575 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.378727 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.378879 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.379026 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.379166 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.379314 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.379455 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.379595 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.379742 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.379891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.380040 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.380193 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.380334 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.380482 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.380622 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.380771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.380911 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.381067 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.381214 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.381354 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.381501 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.381641 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.381780 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.381928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.382069 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.382711 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.382872 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.383020 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.383178 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.383325 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.383464 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.383603 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.383765 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.383915 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.384489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.384993 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.385159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.385310 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.385459 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.385744 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.385913 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.386069 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.386218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.386368 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.386595 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.386747 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.386887 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.387026 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.387173 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.387322 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.387463 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.387647 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.387647 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.387961 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.388110 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.388258 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.388455 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.388611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.388759 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.388909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.389049 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.389204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.389358 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.389499 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.389639 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.389784 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.389943 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.390085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.390233 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.390382 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.390559 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.390707 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.390846 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.391064 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.391218 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.391368 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.391523 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.391664 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.391811 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.391977 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.392120 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:44:13.392259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.392399 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.392546 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.392687 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.392841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.392984 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.393132 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.393272 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.393412 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.393551 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.393714 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.393855 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.393995 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.394142 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.394721 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.395255 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.395416 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.395565 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.395706 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.395706 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.395979 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.396136 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.396285 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.396447 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.396595 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.396595 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.396870 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.396870 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.397166 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.397307 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.397447 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.397595 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.397745 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.397925 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.398073 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.398214 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.398373 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.398637 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.398792 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.398934 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.399074 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.399221 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.399379 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.399527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.399677 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.399826 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.399966 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.400113 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.400254 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.400420 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.400578 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.400719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.400859 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.400998 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.401137 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.401291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.401441 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.401582 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.401732 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.401873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.402012 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.402176 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.402324 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.402488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.402639 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.402796 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.402945 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.403098 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.403239 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.403385 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.403525 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.403664 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.403803 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.403956 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.404107 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.404246 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.404410 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.405002 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.405562 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.405719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.405886 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.406160 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.406319 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.406520 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.406682 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.406831 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.406978 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.407131 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.407280 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.407421 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.407570 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.407719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.407867 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.408008 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.408147 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.408287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.408435 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.408632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.408792 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.408940 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.409103 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.409244 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.409393 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.409542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.409690 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.409831 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.409971 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.410120 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.410261 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.410418 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.410564 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.410705 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.410855 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.411009 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.411157 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.411297 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.411444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.411592 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.411739 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.411894 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.412035 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.412176 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.412315 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.412461 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.412608 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.412770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.412920 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.413060 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.413199 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.413339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.413530 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.413680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.413820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.413970 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.414109 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.414259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.414426 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.414602 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.415749 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.415749 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.416048 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.416189 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.416349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.416498 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.416639 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.416787 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.416928 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.417068 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.417219 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.417360 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.417507 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.417659 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.417799 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.417954 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.418104 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.418246 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.418386 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.418687 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.418836 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.418999 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.419141 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.419282 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.419430 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.419571 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.419728 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.419882 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.420022 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.420169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.420317 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.420465 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.420605 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.420605 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.420891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.421038 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.421178 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.421488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.421729 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.421881 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.422029 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.422168 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.422314 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.422488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.422649 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.422792 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.422938 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.423079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.423369 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.423532 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.423690 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.423845 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.423985 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.424125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.424264 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.424404 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.424558 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.424707 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.424874 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.425354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.426056 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.426219 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.426370 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.426535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.426699 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.426840 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.426988 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.427137 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.427286 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.427428 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.427568 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.427708 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.427860 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.428010 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.428161 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.428310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.428458 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.428616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.428808 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.428969 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.429636 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.429780 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.429927 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.430068 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.430208 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.430357 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.430636 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.430780 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.430927 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.431068 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.431228 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.431371 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.431510 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.431649 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.431796 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.431946 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.432104 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.432244 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.432390 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.432537 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.432677 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.432826 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.432988 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.433129 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.433269 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.433414 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.433562 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.433701 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.433857 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.434005 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.434146 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.434292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.434456 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.434650 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.434794 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.434952 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.435119 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.435649 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.436451 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.436625 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.436780 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.436930 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.437097 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.437239 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.437387 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.437542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.437690 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.437831 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.437989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.438141 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.438289 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.438462 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.438616 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.438765 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.438954 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.439100 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.439250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.439393 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.439541 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.439689 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.439689 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.439992 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.440153 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.440297 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.440461 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.440626 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.440626 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.440962 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.441126 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.443708 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.443886 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.443886 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.444167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.444167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.444466 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.444466 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.444807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.445854 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.446512 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.446678 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.446827 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.446981 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.447145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.447303 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.447449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.447596 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.447753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.447908 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.448055 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.448210 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.448371 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.448528 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.448684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.448831 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.449026 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.449194 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.449351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.449351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.449652 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.449833 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.450005 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.450161 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.450308 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.450536 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.450693 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.450848 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.451048 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.451198 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.451351 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.451351 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.451630 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.451630 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.451935 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.452084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.452230 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.452392 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.452541 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.452702 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.452851 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.452998 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.453145 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.453298 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.453454 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.453620 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.453768 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.453925 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.454072 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.454227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.454374 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.454691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.454850 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.455006 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.455154 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.455301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.455461 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.455985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.456436 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.456436 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.457004 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.457171 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.457330 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.457330 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.457635 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.457635 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.457925 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.457925 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.458219 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.458374 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.458615 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.458768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.458932 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.459129 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.459361 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.459520 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.459668 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.459813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.459959 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.460114 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.460302 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.460466 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.460621 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.460768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.460914 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.461084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.461233 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.461393 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.461548 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.461705 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.461859 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.462019 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.462168 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.462315 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.462495 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.462646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.462801 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.462972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.463121 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.463268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.463422 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.463584 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.463740 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.463906 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.463906 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.464193 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.464349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.464497 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.464657 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.464805 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.464968 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.465123 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.465270 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.465437 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.465595 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.466159 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.466606 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.466885 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.467165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.467331 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.467488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.467636 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.467783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.467955 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.467955 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.468244 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.468399 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.468399 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.468687 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.468845 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.468845 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.469228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.469382 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.469382 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.469671 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.469828 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.469828 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.470170 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.470323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.470323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.470640 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.470797 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.470797 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.471084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.471230 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.471230 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.471531 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.471688 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.471688 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.471976 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.472122 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.472122 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.472413 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.472561 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.472561 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.472847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.472995 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.472995 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.473281 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.473429 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.473576 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.473740 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.473896 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.474051 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.474207 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.474362 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.474564 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.474711 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.474857 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.475025 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.475180 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.475341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.475493 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.475640 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.475801 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.476290 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.476467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.476835 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.476835 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.477340 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.477497 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.477654 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.477809 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.477958 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.478104 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.478251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.478414 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.478597 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.478746 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.478902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.479051 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.479240 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.479410 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.479559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.479706 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.479852 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.480000 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.480153 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.480316 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.480474 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.480620 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.480773 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.480934 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.480934 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.481229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.481385 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.481532 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.481686 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.481832 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.481979 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.482136 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.482293 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.482467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.482620 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.482777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.482934 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.483081 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.483228 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.483375 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.483521 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.483676 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.483832 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.483980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.484152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.484411 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.484600 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.484760 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.484917 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.485072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.485220 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.485373 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.485533 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.485706 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.485958 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.486495 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.486667 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.487061 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.487338 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.487611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.487770 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.487924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.488080 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.488235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.488383 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.488543 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.488700 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.488855 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.489011 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.489157 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.489351 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.489521 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.489688 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.489688 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.489970 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.490116 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.490269 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.490503 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.490659 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.490819 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.490974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.491121 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.491277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.491431 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.492110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.492270 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.492426 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.492583 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.492743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.492900 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.493047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.493194 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.493347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.493495 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.493657 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.493805 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.493951 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.494114 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.494114 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.494510 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.494685 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.494835 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.494982 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.495136 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.495282 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.495453 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.495610 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.495764 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.495911 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.496065 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.496248 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.496743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.496894 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.497327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.497601 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.497854 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.498011 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.498238 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.498415 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.498571 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.498718 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.498877 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.499025 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.499198 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.499347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.499550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.499706 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.499867 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.500099 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.500274 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.500424 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.500586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.500739 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.500885 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.501044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.501210 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.501357 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.501512 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.501660 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.501806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.501962 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.502111 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.502266 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.502266 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.502621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.502802 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.502802 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.503101 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.503278 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.503440 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.504260 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.504418 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.504567 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.504567 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.504866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.505024 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.505180 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.505334 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.505494 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.505657 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.505814 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.505961 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.505961 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.506419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.506980 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.508240 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.508417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.508567 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.508713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.508860 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.509014 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.509178 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.509353 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.509559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.509714 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.509860 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.510020 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.510020 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.510319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.510319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.510657 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.510816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.510964 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.511120 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.511268 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.511268 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.511566 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.511713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.511873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.512020 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.512174 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.512327 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.512474 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.512630 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.512787 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.512935 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.513088 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.513272 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.513444 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.513607 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.513755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.513901 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.514047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.514201 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.514357 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.514543 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.514701 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.514848 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.515003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.515151 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.515317 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.515484 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.515484 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.515789 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.515962 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.516161 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.516322 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.516478 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.516626 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.517215 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.517742 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.517873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.518075 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.518657 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.518943 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.519041 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.519141 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.519239 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.519386 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.519498 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.519498 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.519819 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.519819 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.520001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.520088 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.520171 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.520262 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.520346 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.520429 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.520525 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.520610 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.520700 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.520790 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.520886 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.520973 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.521066 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.521167 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.521263 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.521347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.521441 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.521602 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.521696 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.521782 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.521782 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.521935 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.522019 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.522111 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.522197 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.522290 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.522381 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.522498 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.522584 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.522584 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.522584 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.522584 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.522933 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.523018 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.523119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.523268 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.523365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.523450 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.523534 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.523624 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.523718 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.523803 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.523894 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.523986 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.523986 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.524147 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.524241 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.524335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.524335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.524497 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.524497 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.524665 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.524758 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.524850 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.524940 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.525033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.525118 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.525208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.525324 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.525470 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.525599 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.525718 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.525846 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.525955 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.526049 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.526135 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.526219 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.526308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.526437 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.526537 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.526634 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.526768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.526940 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.527348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.527452 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.527547 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.527708 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.527838 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.527934 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.528026 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.528119 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.528216 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.528327 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.528459 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.528626 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.528722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.528817 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.528903 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.528994 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.529078 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.529164 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.529237 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.529316 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.529389 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.529465 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.529537 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.529627 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.529887 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.529995 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.530082 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.530165 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.530247 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.530337 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.530467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.530555 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.530647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.530739 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.530824 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.530906 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.531002 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.531158 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.531158 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.531158 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.531434 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.531518 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.531518 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.531681 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.531765 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.531848 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.531939 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.532024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.532115 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.532200 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.532282 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.532372 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.532456 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.532547 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.532638 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.532722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.532813 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.532896 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.532987 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.533088 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.533201 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.533201 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.533361 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.533459 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.533544 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.533637 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.533721 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.533804 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.533886 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.533975 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.534060 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.534156 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.534250 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.534335 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.534484 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.534584 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.534684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.534770 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.534860 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.534944 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.535027 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.535118 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.535118 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.535281 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.535281 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.535446 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.535535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.535674 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.535767 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.535859 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.535945 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.536035 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.536125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.536218 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.536309 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.536401 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.536495 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.536586 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.536671 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.536671 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.536844 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.536935 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.537026 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.537145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.537354 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.537458 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.537545 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.537629 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.537718 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.537791 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.537870 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.537949 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.538021 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.538021 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.538190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.538275 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.538371 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.538547 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.538659 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.538777 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.538905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.539001 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.539001 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.539170 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.539254 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.539337 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.539420 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.539584 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.539680 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.539779 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.539881 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.539966 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.540305 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.540407 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.540481 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.540551 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.540641 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.540727 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.540820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.540906 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.540995 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.541087 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.541182 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.541275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.541412 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.541507 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.541599 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.541698 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.541790 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.541875 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.541967 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.542095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.542185 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.542269 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.542360 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.542660 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.542765 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.542853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.542944 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.542944 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.543113 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.543211 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.543311 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.543447 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.543559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.543676 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.543787 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.543905 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.544024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.544134 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.544241 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.544349 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.544455 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.544573 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.544689 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.544799 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.544914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.545024 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.545024 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.545236 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.545236 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.545236 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.545558 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.545667 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.545789 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.545899 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.546008 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.546124 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.546233 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.546348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.546496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.546593 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.546679 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.546762 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.546853 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.546951 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.547044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.547130 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.547220 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.547345 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.547454 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.547454 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.547454 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.547454 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.547865 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.547961 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.548069 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.548234 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.548567 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.548656 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.548990 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.549096 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.549211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.549313 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.549532 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.549623 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.549848 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.549947 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.550172 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.550273 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.550586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.550725 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.550949 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.551185 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.551278 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.551362 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.551556 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.551660 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.551883 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.551989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.552186 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.552301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.552527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.552715 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.552869 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.552983 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.553188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.553288 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.553500 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.553599 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.553831 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.553929 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.554699 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.554828 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.555005 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.555100 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.555337 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.555441 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.555650 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.555742 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.555958 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.556051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.556570 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.556690 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.556777 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.557025 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.557132 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.557335 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.557442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.557642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.557899 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.558021 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.558096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.558349 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.558494 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.558585 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.558765 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.559059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.559157 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.559285 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.559393 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.559671 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.559770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.560038 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.560132 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.560257 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.560384 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.560560 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.560673 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.560888 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.560990 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.561193 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.561297 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.561498 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.561664 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.561816 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.561916 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.562126 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.562233 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.562472 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.562581 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.562789 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.562882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.563100 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.563241 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.563451 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.563544 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.563763 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.563854 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.564078 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.564322 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.564414 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.564657 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.564756 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.564977 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.565079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.565315 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.565407 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.565646 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.565737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.565952 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.566047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.566258 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.566357 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.566589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.566729 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.566904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.566996 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.567216 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.567313 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.567529 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.567621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.567840 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.567976 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.568157 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.568251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.568471 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.568597 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.570154 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.570309 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.570451 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.570574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.570685 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.570817 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.571001 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.571229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.571444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.571565 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.571682 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.571803 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.571922 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.572089 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.572208 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.572208 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.572478 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.572600 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.572726 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.572901 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.573019 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.573135 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.573277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.573390 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.573497 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.573645 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.573765 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.573892 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.574061 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.574185 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.574350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.574560 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.574711 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.575364 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.575458 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.575552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.575636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.575728 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.575826 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.575919 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.576012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.576097 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.576188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.576272 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.576368 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.576455 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.576546 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.576637 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.576728 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.576867 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.576974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.577059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.577160 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.577261 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.577353 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.577556 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.577665 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.577880 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.577973 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.578190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.578291 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.578656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.578765 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.578865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.578963 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.579117 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.579254 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.579427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.579529 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.579739 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.579831 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.580045 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.580216 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.580350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.580443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.580648 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.580749 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.580962 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.581068 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.581270 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.581365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.581590 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.581739 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.581884 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.582109 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.582238 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.582476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.582587 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.582812 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.582915 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.583131 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.583223 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.583456 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.583545 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.583758 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.583858 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.584061 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.584154 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.584380 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.584522 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.584670 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.584777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.584971 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.585073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.585795 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.585924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.586105 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.586264 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.586516 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.586623 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.586829 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.587029 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.587188 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.587371 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.587490 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.587694 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.587819 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.588016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.588133 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.588351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.588447 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.588650 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.588777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.588980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.589086 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.589310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.589452 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.589625 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.589730 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.589948 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.590049 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.590259 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.590358 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.590582 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.590683 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.590899 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.591004 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.591221 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.591320 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.591535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.591635 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.591848 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.591939 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.592158 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.592251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.592471 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.592561 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.592784 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.592882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.593091 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.593183 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.593401 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.593492 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.593709 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.593810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.594015 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.594107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.594323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.594501 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.594690 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.594792 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.595024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.595116 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.595334 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.595592 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.595699 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.595794 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.595965 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.596066 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.596285 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.596387 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.596601 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.596701 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.596911 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.597013 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.597240 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.597344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.597549 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.597857 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.597955 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.598171 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.598278 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.598437 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127030) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 28 09:44:13.598583 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.598806 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.599119 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.599814 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.599814 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.600037 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.600354 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.600673 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.601053 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.601363 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.601686 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.602008 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.602358 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.602710 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.603058 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.603381 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.603693 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.604090 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.604303 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:44:13.604542 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127029) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 28 09:44:13.661407 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO dbcounter [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] Registered counter for database nova_api Oct 28 09:44:13.665920 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO dbcounter [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] Registered counter for database nova_api Oct 28 09:44:13.683016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_db.sqlalchemy.engines [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127030) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:44:13.686327 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_db.sqlalchemy.engines [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127029) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:44:13.690314 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writer thread running {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:44:13.693113 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writer thread running {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:44:13.751251 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.context [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),7c09c1b6-7514-40b3-bb3f-05d78565b59d(cell1) {{(pid=127029) load_cells /opt/stack/nova/nova/context.py:464}} Oct 28 09:44:13.751521 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.context [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),7c09c1b6-7514-40b3-bb3f-05d78565b59d(cell1) {{(pid=127030) load_cells /opt/stack/nova/nova/context.py:464}} Oct 28 09:44:13.755176 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:44:13.755323 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:44:13.755981 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:44:13.756275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:44:13.757228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:44:13.757351 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:44:13.757963 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:44:13.758070 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:44:13.758422 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:44:13.758541 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:44:13.760438 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:44:13.760554 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:44:13.762463 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO dbcounter [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] Registered counter for database nova_cell0 Oct 28 09:44:13.762590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO dbcounter [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] Registered counter for database nova_cell0 Oct 28 09:44:13.766563 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO dbcounter [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] Registered counter for database nova_cell1 Oct 28 09:44:13.766704 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO dbcounter [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] Registered counter for database nova_cell1 Oct 28 09:44:13.775218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_db.sqlalchemy.engines [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127030) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:44:13.776316 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_db.sqlalchemy.engines [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127029) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:44:13.778021 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_db.sqlalchemy.engines [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127030) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:44:13.779810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_db.sqlalchemy.engines [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127029) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:44:13.782820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writer thread running {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:44:13.784720 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writer thread running {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:44:13.786038 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writer thread running {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:44:13.925997 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writer thread running {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:44:13.935301 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO dbcounter [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] Registered counter for database nova_cell0 Oct 28 09:44:13.936228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO dbcounter [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] Registered counter for database nova_cell0 Oct 28 09:44:13.942377 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_db.sqlalchemy.engines [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127029) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:44:13.944134 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_db.sqlalchemy.engines [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127030) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:44:13.949486 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writer thread running {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:44:13.950194 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writer thread running {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:44:14.273851 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: WARNING keystonemiddleware.auth_token [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 09:44:14.282209 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: WARNING keystonemiddleware.auth_token [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 09:44:14.282351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: WARNING keystonemiddleware.auth_token [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 09:44:14.289044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: WARNING keystonemiddleware.auth_token [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 09:44:14.634475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: WARNING keystonemiddleware.auth_token [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 09:44:14.640114 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: WARNING keystonemiddleware.auth_token [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 09:44:14.640703 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: WARNING keystonemiddleware.auth_token [None req-5ea473fa-e221-411d-9471-4f2ef4ea973c None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 09:44:14.646049 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e864e319c0 pid: 127030 (default app) Oct 28 09:44:14.646231 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: WARNING keystonemiddleware.auth_token [None req-3377b449-e94d-42fe-b5f2-bbb4b6831e39 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 09:44:14.652249 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e864e319c0 pid: 127029 (default app) Oct 28 09:44:52.411194 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=3 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:44:52.412084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=3 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:44:52.412443 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=3 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:44:52.417529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=3 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:44:54.244354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-aa31a2e4-8911-4475-a14a-5cf2f596499b tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Action: 'create', calling method: .version_select at 0x7f271b4d7310>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-577836679"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:44:55.850348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-aa31a2e4-8911-4475-a14a-5cf2f596499b tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 3.458843 Oct 28 09:44:55.851284 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 1/1] 10.222.0.25 () {60 vars in 1155 bytes} [Fri Oct 28 09:44:52 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 3461 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 28 09:44:55.872709 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=3 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:44:55.874658 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=3 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:44:55.875919 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=3 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:44:55.891955 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=3 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:44:56.325731 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d29eae6d-747d-4527-955e-e08ad8c7e5e9 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b41f040>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:44:56.975661 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d29eae6d-747d-4527-955e-e08ad8c7e5e9 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/2cce7d54-6342-4fd8-8c52-7ae9093d14f4" status: 200 len: 345 microversion: 2.1 time: 1.110282 Oct 28 09:44:56.977153 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 1/2] 10.222.0.25 () {58 vars in 1245 bytes} [Fri Oct 28 09:44:55 2022] GET /compute/v2.1/os-volumes/2cce7d54-6342-4fd8-8c52-7ae9093d14f4 => generated 345 bytes in 1113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:44:56.992242 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-947c4f1c-718c-4ef4-a64d-de63648cc92e tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Action: 'create', calling method: .version_select at 0x7f271b2fb160>, body: {"snapshot": {"volume_id": "2cce7d54-6342-4fd8-8c52-7ae9093d14f4", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-812775307"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:44:57.392679 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-947c4f1c-718c-4ef4-a64d-de63648cc92e tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.406208 Oct 28 09:44:57.399710 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 2/3] 10.222.0.25 () {60 vars in 1162 bytes} [Fri Oct 28 09:44:56 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:44:57.408440 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-69b2e9e5-1da5-4708-ae85-4afb0f53e20f tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b2e3280>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:44:57.508263 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-69b2e9e5-1da5-4708-ae85-4afb0f53e20f tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6" status: 200 len: 286 microversion: 2.1 time: 0.104944 Oct 28 09:44:57.509024 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1251 bytes} [Fri Oct 28 09:44:57 2022] GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6 => generated 286 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:44:57.522469 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-35004ea7-d033-4137-b710-26f71d0303fe tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b2ff430>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:44:57.631899 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-35004ea7-d033-4137-b710-26f71d0303fe tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.113264 Oct 28 09:44:57.633517 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 3/5] 10.222.0.25 () {58 vars in 1140 bytes} [Fri Oct 28 09:44:57 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:44:57.646989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-331980d6-2ef3-4e64-a791-a49eedb7eafb tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b2e39d0>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:44:57.688635 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-331980d6-2ef3-4e64-a791-a49eedb7eafb tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6" status: 200 len: 286 microversion: 2.1 time: 0.046897 Oct 28 09:44:57.689752 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 3/6] 10.222.0.25 () {58 vars in 1251 bytes} [Fri Oct 28 09:44:57 2022] GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6 => generated 286 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:44:58.708614 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0fc88217-1f55-486e-97dd-dcd35c76c826 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b2ffc10>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:44:58.798341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0fc88217-1f55-486e-97dd-dcd35c76c826 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6" status: 200 len: 286 microversion: 2.1 time: 0.097272 Oct 28 09:44:58.798970 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1251 bytes} [Fri Oct 28 09:44:58 2022] GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6 => generated 286 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:44:59.811885 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-cc11759e-9d84-473d-a58f-6cb892b39fa2 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b2e3c10>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:44:59.874440 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-cc11759e-9d84-473d-a58f-6cb892b39fa2 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6" status: 200 len: 286 microversion: 2.1 time: 0.066386 Oct 28 09:44:59.892513 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1251 bytes} [Fri Oct 28 09:44:59 2022] GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6 => generated 286 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:45:00.902967 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ea22eb8b-7ffa-4a08-b186-ae7e54bd8298 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b2ff310>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:00.962261 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ea22eb8b-7ffa-4a08-b186-ae7e54bd8298 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6" status: 200 len: 287 microversion: 2.1 time: 0.066038 Oct 28 09:45:00.964034 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 5/9] 10.222.0.25 () {58 vars in 1251 bytes} [Fri Oct 28 09:45:00 2022] GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6 => generated 287 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:45:00.977974 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-340dcfd9-296a-4e2b-a17e-a3a4045a2cb4 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b2e3430>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:01.158115 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-340dcfd9-296a-4e2b-a17e-a3a4045a2cb4 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "DELETE /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6" status: 202 len: 0 microversion: 2.1 time: 0.184147 Oct 28 09:45:01.161663 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 5/10] 10.222.0.25 () {58 vars in 1254 bytes} [Fri Oct 28 09:45:00 2022] DELETE /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6 => generated 0 bytes in 186 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:45:01.172849 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2e1ace4f-9d68-479c-851e-8e2556201f27 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b2fbd30>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:01.273880 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2e1ace4f-9d68-479c-851e-8e2556201f27 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6" status: 200 len: 286 microversion: 2.1 time: 0.108670 Oct 28 09:45:01.276507 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 6/11] 10.222.0.25 () {58 vars in 1251 bytes} [Fri Oct 28 09:45:01 2022] GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6 => generated 286 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:45:02.290075 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-49b66fa6-ec18-4a9c-8226-0da975938d50 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b2f1160>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:02.352105 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.wsgi [None req-49b66fa6-ec18-4a9c-8226-0da975938d50 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] HTTP exception thrown: Snapshot b25c5e95-3ce4-419f-bab6-10b023fb96c6 could not be found. Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-49b66fa6-ec18-4a9c-8226-0da975938d50 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Request method failure captured: Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: request: GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6 HTTP/1.1 Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept: application/json Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept-Encoding: identity Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Connection: close Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Length: 0 Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Type: application/json Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Host: 10.222.0.25 Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Openstack-System-Scope: None Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: User-Agent: python-urllib3/1.26.12 Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Auth-Token: gAAAAABjW6SUl2kQOVvHGo-GlcPet-KQ2XqMpIxhcRBUHoeYGLZdC6s9dGhoVizQ7GOYvBsMdxUsY3C9nIhdQQy51fxBhthu-EVtLwvN8k3z5zPEUt9GAETR7HCnjTP9xNwgqDRAw597nZVxeHxq2QVQI9FfVte4MepPPFj5Dz8gtpYXGIAMzbE Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Id: None Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Name: None Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Identity-Status: Confirmed Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Is-Admin-Project: True Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Id: default Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Name: Default Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Id: 1d65871da7a445fda2f75ea82af5f9b9 Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1970743202 Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Role: member,reader Oct 28 09:45:02.359152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Roles: member,reader Oct 28 09:45:02.360309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1d65871da7a445fda2f75ea82af5f9b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1d65871da7a445fda2f75ea82af5f9b9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1d65871da7a445fda2f75ea82af5f9b9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1d65871da7a445fda2f75ea82af5f9b9"}]}] Oct 28 09:45:02.360309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1970743202 Oct 28 09:45:02.360309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Id: 1d65871da7a445fda2f75ea82af5f9b9 Oct 28 09:45:02.360309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1970743202 Oct 28 09:45:02.360309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User: tempest-VolumesSnapshotsTestJSON-1970743202-project Oct 28 09:45:02.360309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Id: default Oct 28 09:45:02.360309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Name: Default Oct 28 09:45:02.360309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Id: f63cfb4896ff45848bbd79a65ef0cdf1 Oct 28 09:45:02.360309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1970743202-project Oct 28 09:45:02.360309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: method: Oct 28 09:45:02.360309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: exception: Snapshot b25c5e95-3ce4-419f-bab6-10b023fb96c6 could not be found. Oct 28 09:45:02.360309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: action_args: {'id': 'b25c5e95-3ce4-419f-bab6-10b023fb96c6'} Oct 28 09:45:02.360309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot b25c5e95-3ce4-419f-bab6-10b023fb96c6 could not be found. (HTTP 404) (Request-ID: req-9f80a124-d853-4066-ac0e-472868fa7ab3) Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:02.361055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 09:45:02.361918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot b25c5e95-3ce4-419f-bab6-10b023fb96c6 could not be found. Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Oct 28 09:45:02.363287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:45:02.364417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot b25c5e95-3ce4-419f-bab6-10b023fb96c6 could not be found. Oct 28 09:45:02.364417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:02.364417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:45:02.364417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:02.364417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:02.364417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:45:02.364417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:45:02.364417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:45:02.364417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:45:02.364417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot b25c5e95-3ce4-419f-bab6-10b023fb96c6 could not be found. Oct 28 09:45:02.364417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:02.364417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-49b66fa6-ec18-4a9c-8226-0da975938d50 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Returning 404 to user: Snapshot b25c5e95-3ce4-419f-bab6-10b023fb96c6 could not be found. {{(pid=127029) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:45:02.364417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-49b66fa6-ec18-4a9c-8226-0da975938d50 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6" status: 404 len: 111 microversion: 2.1 time: 0.077678 Oct 28 09:45:02.366783 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1251 bytes} [Fri Oct 28 09:45:02 2022] GET /compute/v2.1/os-snapshots/b25c5e95-3ce4-419f-bab6-10b023fb96c6 => generated 111 bytes in 82 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:45:02.375715 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5f2fba62-30d2-4a27-8d6d-00495f3e8b22 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b2ff310>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:02.620415 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5f2fba62-30d2-4a27-8d6d-00495f3e8b22 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/2cce7d54-6342-4fd8-8c52-7ae9093d14f4" status: 202 len: 0 microversion: 2.1 time: 0.249280 Oct 28 09:45:02.621430 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1248 bytes} [Fri Oct 28 09:45:02 2022] DELETE /compute/v2.1/os-volumes/2cce7d54-6342-4fd8-8c52-7ae9093d14f4 => generated 0 bytes in 251 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:45:02.629705 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-433dbe4c-3b82-4f23-8ca9-e93bd176f895 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b2f1b80>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:03.133374 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.wsgi [None req-433dbe4c-3b82-4f23-8ca9-e93bd176f895 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] HTTP exception thrown: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-433dbe4c-3b82-4f23-8ca9-e93bd176f895 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Request method failure captured: Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: request: GET /compute/v2.1/os-volumes/2cce7d54-6342-4fd8-8c52-7ae9093d14f4 HTTP/1.1 Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept: application/json Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept-Encoding: identity Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Connection: close Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Length: 0 Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Type: application/json Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Host: 10.222.0.25 Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Openstack-System-Scope: None Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: User-Agent: python-urllib3/1.26.12 Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Auth-Token: gAAAAABjW6SUl2kQOVvHGo-GlcPet-KQ2XqMpIxhcRBUHoeYGLZdC6s9dGhoVizQ7GOYvBsMdxUsY3C9nIhdQQy51fxBhthu-EVtLwvN8k3z5zPEUt9GAETR7HCnjTP9xNwgqDRAw597nZVxeHxq2QVQI9FfVte4MepPPFj5Dz8gtpYXGIAMzbE Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Id: None Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Name: None Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Identity-Status: Confirmed Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Is-Admin-Project: True Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Id: default Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Name: Default Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Id: 1d65871da7a445fda2f75ea82af5f9b9 Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1970743202 Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Role: member,reader Oct 28 09:45:03.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Roles: member,reader Oct 28 09:45:03.142956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1d65871da7a445fda2f75ea82af5f9b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1d65871da7a445fda2f75ea82af5f9b9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1d65871da7a445fda2f75ea82af5f9b9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1d65871da7a445fda2f75ea82af5f9b9"}]}] Oct 28 09:45:03.142956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1970743202 Oct 28 09:45:03.142956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Id: 1d65871da7a445fda2f75ea82af5f9b9 Oct 28 09:45:03.142956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1970743202 Oct 28 09:45:03.142956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User: tempest-VolumesSnapshotsTestJSON-1970743202-project Oct 28 09:45:03.142956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Id: default Oct 28 09:45:03.142956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Name: Default Oct 28 09:45:03.142956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Id: f63cfb4896ff45848bbd79a65ef0cdf1 Oct 28 09:45:03.142956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1970743202-project Oct 28 09:45:03.142956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: method: Oct 28 09:45:03.142956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: exception: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.142956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: action_args: {'id': '2cce7d54-6342-4fd8-8c52-7ae9093d14f4'} Oct 28 09:45:03.142956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. (HTTP 404) (Request-ID: req-fcad6bfc-fc05-4239-ac5a-e9e4b9471ea5) Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:45:03.144281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 09:45:03.146169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 28 09:45:03.148250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:45:03.149246 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.149246 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.149246 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:45:03.149246 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.149246 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:03.149246 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:45:03.149246 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:45:03.149246 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:45:03.149246 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:45:03.149246 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.149246 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.151225 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-433dbe4c-3b82-4f23-8ca9-e93bd176f895 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Returning 404 to user: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. {{(pid=127029) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:45:03.152005 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-433dbe4c-3b82-4f23-8ca9-e93bd176f895 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/2cce7d54-6342-4fd8-8c52-7ae9093d14f4" status: 404 len: 109 microversion: 2.1 time: 0.526511 Oct 28 09:45:03.152565 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 7/14] 10.222.0.25 () {58 vars in 1245 bytes} [Fri Oct 28 09:45:02 2022] GET /compute/v2.1/os-volumes/2cce7d54-6342-4fd8-8c52-7ae9093d14f4 => generated 109 bytes in 528 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:45:03.166922 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4737a2c9-b49b-4708-92b8-0b0014754bd3 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b2ffc10>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:03.586623 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-4737a2c9-b49b-4708-92b8-0b0014754bd3 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] HTTP exception thrown: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4737a2c9-b49b-4708-92b8-0b0014754bd3 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Request method failure captured: Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: DELETE /compute/v2.1/os-volumes/2cce7d54-6342-4fd8-8c52-7ae9093d14f4 HTTP/1.1 Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6SUl2kQOVvHGo-GlcPet-KQ2XqMpIxhcRBUHoeYGLZdC6s9dGhoVizQ7GOYvBsMdxUsY3C9nIhdQQy51fxBhthu-EVtLwvN8k3z5zPEUt9GAETR7HCnjTP9xNwgqDRAw597nZVxeHxq2QVQI9FfVte4MepPPFj5Dz8gtpYXGIAMzbE Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 1d65871da7a445fda2f75ea82af5f9b9 Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1970743202 Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:45:03.596882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:45:03.598165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1d65871da7a445fda2f75ea82af5f9b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1d65871da7a445fda2f75ea82af5f9b9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1d65871da7a445fda2f75ea82af5f9b9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1d65871da7a445fda2f75ea82af5f9b9"}]}] Oct 28 09:45:03.598165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1970743202 Oct 28 09:45:03.598165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 1d65871da7a445fda2f75ea82af5f9b9 Oct 28 09:45:03.598165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1970743202 Oct 28 09:45:03.598165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-VolumesSnapshotsTestJSON-1970743202-project Oct 28 09:45:03.598165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:45:03.598165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:45:03.598165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: f63cfb4896ff45848bbd79a65ef0cdf1 Oct 28 09:45:03.598165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1970743202-project Oct 28 09:45:03.598165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: Oct 28 09:45:03.598165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.598165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '2cce7d54-6342-4fd8-8c52-7ae9093d14f4'} Oct 28 09:45:03.598165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. (HTTP 404) (Request-ID: req-69b8c27d-fb3d-4b59-838a-35b96b3e5d39) Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:45:03.598948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 28 09:45:03.600505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 28 09:45:03.601364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:45:03.602208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.602208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.602208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:45:03.602208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.602208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:03.602208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:45:03.602208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:45:03.602208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:45:03.602208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:45:03.602208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.602208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.602208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4737a2c9-b49b-4708-92b8-0b0014754bd3 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Returning 404 to user: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:45:03.602208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4737a2c9-b49b-4708-92b8-0b0014754bd3 tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/2cce7d54-6342-4fd8-8c52-7ae9093d14f4" status: 404 len: 109 microversion: 2.1 time: 0.435886 Oct 28 09:45:03.604859 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 8/15] 10.222.0.25 () {58 vars in 1248 bytes} [Fri Oct 28 09:45:03 2022] DELETE /compute/v2.1/os-volumes/2cce7d54-6342-4fd8-8c52-7ae9093d14f4 => generated 109 bytes in 443 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:45:03.615353 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3c80a99f-115e-419b-8567-86a980d84aad tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Calling method '.version_select at 0x7f271b2f1940>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:03.691955 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.wsgi [None req-3c80a99f-115e-419b-8567-86a980d84aad tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] HTTP exception thrown: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3c80a99f-115e-419b-8567-86a980d84aad tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Request method failure captured: Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: request: GET /compute/v2.1/os-volumes/2cce7d54-6342-4fd8-8c52-7ae9093d14f4 HTTP/1.1 Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept: application/json Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept-Encoding: identity Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Connection: close Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Length: 0 Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Type: application/json Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Host: 10.222.0.25 Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Openstack-System-Scope: None Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: User-Agent: python-urllib3/1.26.12 Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Auth-Token: gAAAAABjW6SUl2kQOVvHGo-GlcPet-KQ2XqMpIxhcRBUHoeYGLZdC6s9dGhoVizQ7GOYvBsMdxUsY3C9nIhdQQy51fxBhthu-EVtLwvN8k3z5zPEUt9GAETR7HCnjTP9xNwgqDRAw597nZVxeHxq2QVQI9FfVte4MepPPFj5Dz8gtpYXGIAMzbE Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Id: None Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Name: None Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Identity-Status: Confirmed Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Is-Admin-Project: True Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Id: default Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Name: Default Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Id: 1d65871da7a445fda2f75ea82af5f9b9 Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1970743202 Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Role: member,reader Oct 28 09:45:03.695496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Roles: member,reader Oct 28 09:45:03.696822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1d65871da7a445fda2f75ea82af5f9b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1d65871da7a445fda2f75ea82af5f9b9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1d65871da7a445fda2f75ea82af5f9b9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1d65871da7a445fda2f75ea82af5f9b9"}]}] Oct 28 09:45:03.696822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1970743202 Oct 28 09:45:03.696822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Id: 1d65871da7a445fda2f75ea82af5f9b9 Oct 28 09:45:03.696822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1970743202 Oct 28 09:45:03.696822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User: tempest-VolumesSnapshotsTestJSON-1970743202-project Oct 28 09:45:03.696822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Id: default Oct 28 09:45:03.696822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Name: Default Oct 28 09:45:03.696822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Id: f63cfb4896ff45848bbd79a65ef0cdf1 Oct 28 09:45:03.696822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1970743202-project Oct 28 09:45:03.696822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: method: Oct 28 09:45:03.696822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: exception: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.696822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: action_args: {'id': '2cce7d54-6342-4fd8-8c52-7ae9093d14f4'} Oct 28 09:45:03.696822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. (HTTP 404) (Request-ID: req-80c4be04-af04-4653-8313-f662fa8e7500) Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:45:03.697589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 09:45:03.699044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 28 09:45:03.700155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:45:03.701242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.701242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.701242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:45:03.701242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.701242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:45:03.701242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:45:03.701242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:45:03.701242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:45:03.701242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:45:03.701242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. Oct 28 09:45:03.701242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:45:03.703543 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3c80a99f-115e-419b-8567-86a980d84aad tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] Returning 404 to user: Volume 2cce7d54-6342-4fd8-8c52-7ae9093d14f4 could not be found. {{(pid=127029) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:45:03.706286 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-3c80a99f-115e-419b-8567-86a980d84aad tempest-VolumesSnapshotsTestJSON-1970743202 tempest-VolumesSnapshotsTestJSON-1970743202-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/2cce7d54-6342-4fd8-8c52-7ae9093d14f4" status: 404 len: 109 microversion: 2.1 time: 0.094512 Oct 28 09:45:03.711150 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1245 bytes} [Fri Oct 28 09:45:03 2022] GET /compute/v2.1/os-volumes/2cce7d54-6342-4fd8-8c52-7ae9093d14f4 => generated 109 bytes in 101 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:45:08.139267 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1877587627", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:08.397997 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.network.neutron [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] validate_networks() for [('51c62d53-c6c7-4bb9-b1b6-d47428b64a8c', None, None, None, None, None)] {{(pid=127030) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:45:08.816359 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-12269856-0864-41e9-b557-d2d9858d4043 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Action: 'create', calling method: .version_select at 0x7f271b2e39d0>, body: {"keypair": {"name": "tempest-keypair-1471611763", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBI0F0Ikytulrsm/u8QYHokWs7ijFwyvgF1b+YXcgY7zGfwWbusNlWpYCzdBdCGPt1tr/j5ZX+C0SwhrdOIeJkTr6Q0VP+wrG1792Sh3yyDeqjyL8mzSrB1ouXjr1m7GiNQ=="}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:08.982156 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-12269856-0864-41e9-b557-d2d9858d4043 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Getting quotas for project 29c47b13a2044939b5446ffe5397633a. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:45:09.026948 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-12269856-0864-41e9-b557-d2d9858d4043 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Getting quotas for user 715a782e5a56431e88e4fe5a3b0a62aa and project 29c47b13a2044939b5446ffe5397633a. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:45:09.163723 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-12269856-0864-41e9-b557-d2d9858d4043 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.727423 Oct 28 09:45:09.164741 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 9/17] 10.222.0.25 () {62 vars in 1200 bytes} [Fri Oct 28 09:45:08 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 729 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 28 09:45:10.191036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Flavor limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:45:10.192197 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Image limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:45:10.192819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Flavor pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:45:10.196569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Image pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:45:10.199071 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:45:10.261759 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:10.262480 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:10.263264 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:10.265626 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:10.270806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:10.271576 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:10.370254 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Getting quotas for project 227ac6c4e642467ca3b9e0bf7181d5c9. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:45:10.383373 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Getting quotas for user ed9a4c24cbfb4689b7f984219c7503a4 and project 227ac6c4e642467ca3b9e0bf7181d5c9. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:45:10.401507 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Going to run 1 instances... {{(pid=127030) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:45:10.405115 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127030) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:45:10.590724 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-72677860-d6bf-47aa-9a97-9bb8b36606ea tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.738598 Oct 28 09:45:10.593108 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 9/18] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:45:07 2022] POST /compute/v2.1/servers => generated 383 bytes in 2741 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 09:45:10.606013 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-69557af8-5a0d-4f40-8c93-ee869194721c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:10.607371 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-69557af8-5a0d-4f40-8c93-ee869194721c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:11.001306 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-69557af8-5a0d-4f40-8c93-ee869194721c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:11.096157 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-69557af8-5a0d-4f40-8c93-ee869194721c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1222 microversion: 2.1 time: 0.495266 Oct 28 09:45:11.097963 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 10/19] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:10 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1222 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:11.342122 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1479887973", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:11.508016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.network.neutron [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] validate_networks() for [('51c62d53-c6c7-4bb9-b1b6-d47428b64a8c', None, None, None, None, None)] {{(pid=127030) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:45:13.689275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Flavor limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:45:13.690475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Image limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:45:13.691200 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Flavor pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:45:13.692367 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Image pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:45:13.693019 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:45:13.712668 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:13.714381 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:13.715210 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:13.717604 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:13.718418 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:13.719900 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:13.781075 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Getting quotas for project 227ac6c4e642467ca3b9e0bf7181d5c9. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:45:13.803366 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Getting quotas for user ed9a4c24cbfb4689b7f984219c7503a4 and project 227ac6c4e642467ca3b9e0bf7181d5c9. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:45:13.820561 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Going to run 1 instances... {{(pid=127030) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:45:13.823506 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127030) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:45:13.905995 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-230ccd86-116f-44eb-8c28-a3362a05bdb8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.569821 Oct 28 09:45:13.907215 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 10/20] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:45:11 2022] POST /compute/v2.1/servers => generated 383 bytes in 2573 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 09:45:13.919702 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-214b01cc-f33a-421c-bf91-de1c8f730243 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:13.920373 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-214b01cc-f33a-421c-bf91-de1c8f730243 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:14.016768 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-214b01cc-f33a-421c-bf91-de1c8f730243 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:14.035047 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-214b01cc-f33a-421c-bf91-de1c8f730243 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1222 microversion: 2.1 time: 0.119445 Oct 28 09:45:14.036510 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 11/21] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:13 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1222 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:14.075906 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-224426422", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "1c06056f-c33e-47c9-b078-4c3c3f047c82"}], "security_groups": [{"name": "tempest-securitygroup--1448293155"}], "key_name": "tempest-keypair-1471611763", "user_data": "}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:14.298505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.network.neutron [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] validate_networks() for [('1c06056f-c33e-47c9-b078-4c3c3f047c82', None, None, None, None, None)] {{(pid=127030) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:45:15.004362 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-0823117f-46a6-4443-a5ca-1debca8ce1d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Action: 'create', calling method: .version_select at 0x7f271b1c4f70>, body: {"keypair": {"name": "tempest-keypair-428074548", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHi29vS51bEUL4BIH6LkMNPplgGCbJS8QFlVb93DBjir63+WxbxCYf2z+ej9qzS/SVmsXiSC/Ie7MBzFB9qNL0MAx3aKSULFADv5t3arec8/JrX1EUUQlHPJfoGsQ7LVHQ=="}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:15.045331 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-0823117f-46a6-4443-a5ca-1debca8ce1d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for project 4f351d016ace4e369be852bb7aa916fe. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:45:15.050004 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-0823117f-46a6-4443-a5ca-1debca8ce1d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for user e65d298cca0f478eba502edf5bd13954 and project 4f351d016ace4e369be852bb7aa916fe. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:45:15.106276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Flavor limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:45:15.107514 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Image limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:45:15.108143 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Flavor pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:45:15.108693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Image pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:45:15.109254 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:45:15.112601 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-0823117f-46a6-4443-a5ca-1debca8ce1d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.424507 Oct 28 09:45:15.120135 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:15.121729 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 12/22] 10.222.0.25 () {60 vars in 1159 bytes} [Fri Oct 28 09:45:14 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:45:15.126733 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:15.127541 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:15.128502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:15.131577 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:15.132197 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:15.169298 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Getting quotas for project 29c47b13a2044939b5446ffe5397633a. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:45:15.174719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Getting quotas for user 715a782e5a56431e88e4fe5a3b0a62aa and project 29c47b13a2044939b5446ffe5397633a. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:45:15.213711 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Going to run 1 instances... {{(pid=127030) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:45:15.220466 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127030) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:45:15.263583 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-817495607", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "networks": [{"uuid": "375c6b20-2d09-4b21-b297-392868656545"}]}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:15.376518 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3ee0d9e6-e00e-4e18-8e2c-a7dcff42d24f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 1.305001 Oct 28 09:45:15.383122 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 11/23] 10.222.0.25 () {62 vars in 1188 bytes} [Fri Oct 28 09:45:14 2022] POST /compute/v2.1/servers => generated 402 bytes in 1312 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 28 09:45:15.388530 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-944a0d7d-26c7-480d-bb19-52008b699842 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:15.389060 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-944a0d7d-26c7-480d-bb19-52008b699842 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:15.475786 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-944a0d7d-26c7-480d-bb19-52008b699842 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:15.478106 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-944a0d7d-26c7-480d-bb19-52008b699842 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:15.479839 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-944a0d7d-26c7-480d-bb19-52008b699842 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:15.481426 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-944a0d7d-26c7-480d-bb19-52008b699842 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:15.482604 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-944a0d7d-26c7-480d-bb19-52008b699842 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:15.483108 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-944a0d7d-26c7-480d-bb19-52008b699842 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:15.513599 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.network.neutron [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] validate_networks() for [('375c6b20-2d09-4b21-b297-392868656545', None, None, None, None, None)] {{(pid=127029) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:45:16.014008 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-944a0d7d-26c7-480d-bb19-52008b699842 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:16.071429 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-944a0d7d-26c7-480d-bb19-52008b699842 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1226 microversion: 2.1 time: 0.683056 Oct 28 09:45:16.074219 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 12/24] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:15 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1226 bytes in 690 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:16.098331 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1c0e551c-8d64-4c8c-a978-9ada6de06309 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:16.102329 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1c0e551c-8d64-4c8c-a978-9ada6de06309 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:16.403923 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1c0e551c-8d64-4c8c-a978-9ada6de06309 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:16.458899 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1c0e551c-8d64-4c8c-a978-9ada6de06309 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:16.471513 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1c0e551c-8d64-4c8c-a978-9ada6de06309 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:16.473334 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1c0e551c-8d64-4c8c-a978-9ada6de06309 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1281 microversion: 2.20 time: 0.396881 Oct 28 09:45:16.473905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 13/25] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:16 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1281 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:16.728378 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-211f6464-632b-4c62-ade8-c1c2dd4a2aff tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Action: 'create', calling method: .version_select at 0x7f271a898820>, body: {"keypair": {"name": "tempest-keypair-1262841334", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFh89KzuLe3utWmRMcn8ipLkI56beElbo2/kzhYr+7C1Ves/vA+abVkzD2CzqO0aa4VZpq5+Ln7oZdCIlTK+JCVhzcfQjdd5xA9gUeCNy9ipeQaWO9NDrg2eCBbOFpC9rA=="}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:16.831283 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-211f6464-632b-4c62-ade8-c1c2dd4a2aff tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Getting quotas for project f2fcce993e6b4388bde493f0702fd49d. Resources: {'key_pairs'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:45:16.861588 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-211f6464-632b-4c62-ade8-c1c2dd4a2aff tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Getting quotas for user a4237ca76acd4f7483aaf66e1f5d2eea and project f2fcce993e6b4388bde493f0702fd49d. Resources: {'key_pairs'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:45:16.976870 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-211f6464-632b-4c62-ade8-c1c2dd4a2aff tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.498849 Oct 28 09:45:16.979850 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 14/26] 10.222.0.25 () {60 vars in 1159 bytes} [Fri Oct 28 09:45:16 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:45:17.103602 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f553bc5e-022a-4eea-b3e8-a617b2077980 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:17.105789 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f553bc5e-022a-4eea-b3e8-a617b2077980 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:17.144952 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f553bc5e-022a-4eea-b3e8-a617b2077980 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:17.145514 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f553bc5e-022a-4eea-b3e8-a617b2077980 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:17.148913 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f553bc5e-022a-4eea-b3e8-a617b2077980 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:17.149749 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f553bc5e-022a-4eea-b3e8-a617b2077980 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:17.168712 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f553bc5e-022a-4eea-b3e8-a617b2077980 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:17.175779 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f553bc5e-022a-4eea-b3e8-a617b2077980 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:17.376356 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f553bc5e-022a-4eea-b3e8-a617b2077980 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:17.389278 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f553bc5e-022a-4eea-b3e8-a617b2077980 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1274 microversion: 2.1 time: 0.293068 Oct 28 09:45:17.392770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 15/27] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:17 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1274 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:17.501302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f56fafc6-86e6-4775-b43b-b85422e6f723 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:17.502134 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f56fafc6-86e6-4775-b43b-b85422e6f723 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:17.516275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f56fafc6-86e6-4775-b43b-b85422e6f723 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:17.516747 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f56fafc6-86e6-4775-b43b-b85422e6f723 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:17.518357 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f56fafc6-86e6-4775-b43b-b85422e6f723 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:17.519193 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f56fafc6-86e6-4775-b43b-b85422e6f723 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:17.522799 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f56fafc6-86e6-4775-b43b-b85422e6f723 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:17.523260 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f56fafc6-86e6-4775-b43b-b85422e6f723 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:17.841993 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f56fafc6-86e6-4775-b43b-b85422e6f723 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:17.854057 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f56fafc6-86e6-4775-b43b-b85422e6f723 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:17.866992 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f56fafc6-86e6-4775-b43b-b85422e6f723 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:17.868163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f56fafc6-86e6-4775-b43b-b85422e6f723 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1285 microversion: 2.20 time: 0.372406 Oct 28 09:45:17.868712 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 16/28] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:17 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1285 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:18.012525 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Flavor limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:45:18.013171 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Image limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:45:18.013919 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Flavor pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:45:18.014551 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Image pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:45:18.015170 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:45:18.057397 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:18.062698 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:18.063428 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:18.070231 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:18.074264 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:18.081607 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:18.131606 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Getting quotas for project 41516f10d050497e82056ddd1be5981c. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:45:18.145853 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Getting quotas for user e451d264e4964409b09e0b96ae8b983e and project 41516f10d050497e82056ddd1be5981c. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:45:18.162715 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Going to run 1 instances... {{(pid=127029) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:45:18.166096 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127029) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:45:18.345089 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-eb358149-791b-4e1e-bc97-31b973b2031f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.222049 Oct 28 09:45:18.346214 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 13/29] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:45:15 2022] POST /compute/v2.1/servers => generated 380 bytes in 3224 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 09:45:18.360207 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9000bef7-0744-461f-a4c5-34f507c18417 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:18.360810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9000bef7-0744-461f-a4c5-34f507c18417 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:18.419718 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-fb8ea131-4828-49e4-9a17-b576b1c6a618 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:18.420333 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-fb8ea131-4828-49e4-9a17-b576b1c6a618 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:18.434832 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fb8ea131-4828-49e4-9a17-b576b1c6a618 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:18.435327 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fb8ea131-4828-49e4-9a17-b576b1c6a618 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:18.435760 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fb8ea131-4828-49e4-9a17-b576b1c6a618 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:18.436470 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fb8ea131-4828-49e4-9a17-b576b1c6a618 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:18.436915 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fb8ea131-4828-49e4-9a17-b576b1c6a618 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:18.437338 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fb8ea131-4828-49e4-9a17-b576b1c6a618 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:18.569533 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-9000bef7-0744-461f-a4c5-34f507c18417 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:18.586907 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9000bef7-0744-461f-a4c5-34f507c18417 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1214 microversion: 2.1 time: 0.230525 Oct 28 09:45:18.587739 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 17/30] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:18 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1214 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:18.748316 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-fb8ea131-4828-49e4-9a17-b576b1c6a618 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:18.766375 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-fb8ea131-4828-49e4-9a17-b576b1c6a618 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1274 microversion: 2.1 time: 0.350992 Oct 28 09:45:18.775545 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 14/31] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:18 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1274 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:18.900920 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e05a57ac-f015-4b1d-8a17-7c3e38088b90 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:18.901564 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e05a57ac-f015-4b1d-8a17-7c3e38088b90 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:18.924416 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e05a57ac-f015-4b1d-8a17-7c3e38088b90 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:18.925421 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e05a57ac-f015-4b1d-8a17-7c3e38088b90 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:18.927647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e05a57ac-f015-4b1d-8a17-7c3e38088b90 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:18.929188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e05a57ac-f015-4b1d-8a17-7c3e38088b90 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:18.930490 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e05a57ac-f015-4b1d-8a17-7c3e38088b90 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:18.933913 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e05a57ac-f015-4b1d-8a17-7c3e38088b90 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:19.173099 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e05a57ac-f015-4b1d-8a17-7c3e38088b90 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:19.181096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e05a57ac-f015-4b1d-8a17-7c3e38088b90 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:19.182505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e05a57ac-f015-4b1d-8a17-7c3e38088b90 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:19.183641 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e05a57ac-f015-4b1d-8a17-7c3e38088b90 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1277 microversion: 2.20 time: 0.293108 Oct 28 09:45:19.184191 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 18/32] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:18 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1277 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:19.293735 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1611308955", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8cae7e07-cd54-4f98-96b4-a27c37dcb83a"}], "security_groups": [{"name": "tempest-securitygroup--194378170"}], "key_name": "tempest-keypair-428074548", "user_data": "}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:19.477385 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.network.neutron [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] validate_networks() for [('8cae7e07-cd54-4f98-96b4-a27c37dcb83a', None, None, None, None, None)] {{(pid=127029) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:45:19.611524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-fb5e5a43-a545-4caa-bb17-46ea8a82da7e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:19.612138 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-fb5e5a43-a545-4caa-bb17-46ea8a82da7e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:19.622357 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fb5e5a43-a545-4caa-bb17-46ea8a82da7e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:19.622879 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fb5e5a43-a545-4caa-bb17-46ea8a82da7e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:19.623308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fb5e5a43-a545-4caa-bb17-46ea8a82da7e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:19.624007 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fb5e5a43-a545-4caa-bb17-46ea8a82da7e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:19.624448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fb5e5a43-a545-4caa-bb17-46ea8a82da7e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:19.633565 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fb5e5a43-a545-4caa-bb17-46ea8a82da7e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:19.919213 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-fb5e5a43-a545-4caa-bb17-46ea8a82da7e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:19.942000 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-fb5e5a43-a545-4caa-bb17-46ea8a82da7e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1218 microversion: 2.1 time: 0.332466 Oct 28 09:45:19.944684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 19/33] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:19 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1218 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:19.978709 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b71941e9-6612-4ff6-bada-a8f34da46964 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:19.983116 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b71941e9-6612-4ff6-bada-a8f34da46964 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:20.019965 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b71941e9-6612-4ff6-bada-a8f34da46964 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:20.020497 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b71941e9-6612-4ff6-bada-a8f34da46964 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:20.026020 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b71941e9-6612-4ff6-bada-a8f34da46964 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:20.026983 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b71941e9-6612-4ff6-bada-a8f34da46964 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:20.032067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b71941e9-6612-4ff6-bada-a8f34da46964 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:20.032549 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b71941e9-6612-4ff6-bada-a8f34da46964 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:20.265117 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b71941e9-6612-4ff6-bada-a8f34da46964 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:20.276222 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b71941e9-6612-4ff6-bada-a8f34da46964 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1274 microversion: 2.1 time: 0.327206 Oct 28 09:45:20.282186 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 20/34] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:19 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1274 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:20.296277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-10f00402-69cf-459b-8cba-3a3eeec8c09c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:20.297067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-10f00402-69cf-459b-8cba-3a3eeec8c09c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:20.326613 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-10f00402-69cf-459b-8cba-3a3eeec8c09c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:20.327216 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-10f00402-69cf-459b-8cba-3a3eeec8c09c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:20.330049 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-10f00402-69cf-459b-8cba-3a3eeec8c09c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:20.330912 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-10f00402-69cf-459b-8cba-3a3eeec8c09c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:20.331386 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-10f00402-69cf-459b-8cba-3a3eeec8c09c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:20.340115 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-10f00402-69cf-459b-8cba-3a3eeec8c09c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:20.527158 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-10f00402-69cf-459b-8cba-3a3eeec8c09c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:20.536568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-10f00402-69cf-459b-8cba-3a3eeec8c09c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:20.541251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-10f00402-69cf-459b-8cba-3a3eeec8c09c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:20.543472 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-10f00402-69cf-459b-8cba-3a3eeec8c09c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1277 microversion: 2.20 time: 0.259948 Oct 28 09:45:20.549559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 21/35] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:20 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1277 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:20.556067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1918884688", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "1d4760fa-4445-40fa-b7a4-b3e4124ac823"}], "security_groups": [{"name": "tempest-securitygroup--870272728"}], "key_name": "tempest-keypair-1262841334", "user_data": "}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:20.588279 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Flavor limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:45:20.589380 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Image limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:45:20.590056 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Flavor pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:45:20.590703 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Image pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:45:20.591337 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:45:20.619620 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:20.623041 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:20.623728 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:20.625101 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:20.628282 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:20.629007 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:20.677472 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for project 4f351d016ace4e369be852bb7aa916fe. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:45:20.691978 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for user e65d298cca0f478eba502edf5bd13954 and project 4f351d016ace4e369be852bb7aa916fe. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:45:20.720447 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Going to run 1 instances... {{(pid=127029) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:45:20.724313 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127029) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:45:20.727033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.network.neutron [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] validate_networks() for [('1d4760fa-4445-40fa-b7a4-b3e4124ac823', None, None, None, None, None)] {{(pid=127030) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:45:20.826721 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-ab7e125d-cbc1-4753-9f51-6f00cbb71ccb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.538285 Oct 28 09:45:20.828100 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 15/36] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:45:19 2022] POST /compute/v2.1/servers => generated 401 bytes in 1542 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 09:45:20.840003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-039d17ac-0ef6-4402-984d-411ed43c4e97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:20.852119 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-039d17ac-0ef6-4402-984d-411ed43c4e97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:21.478111 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-039d17ac-0ef6-4402-984d-411ed43c4e97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:21.489764 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-039d17ac-0ef6-4402-984d-411ed43c4e97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1241 microversion: 2.1 time: 0.654063 Oct 28 09:45:21.494831 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 16/37] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:20 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1241 bytes in 660 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:21.500061 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2e7f0c16-8a75-48e5-9101-6a2575342504 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:21.502880 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2e7f0c16-8a75-48e5-9101-6a2575342504 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:21.525764 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2e7f0c16-8a75-48e5-9101-6a2575342504 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:21.527195 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2e7f0c16-8a75-48e5-9101-6a2575342504 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:21.527773 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2e7f0c16-8a75-48e5-9101-6a2575342504 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:21.531719 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2e7f0c16-8a75-48e5-9101-6a2575342504 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:21.532206 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2e7f0c16-8a75-48e5-9101-6a2575342504 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:21.532642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2e7f0c16-8a75-48e5-9101-6a2575342504 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:21.737403 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2e7f0c16-8a75-48e5-9101-6a2575342504 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:21.751213 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2e7f0c16-8a75-48e5-9101-6a2575342504 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1210 microversion: 2.1 time: 0.252084 Oct 28 09:45:21.754203 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 17/38] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:21 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1210 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:21.766559 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-9c1c9c18-fde9-4cc9-9686-0f7cc0972aaf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:21.778343 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-9c1c9c18-fde9-4cc9-9686-0f7cc0972aaf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:21.793377 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9c1c9c18-fde9-4cc9-9686-0f7cc0972aaf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:21.805365 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9c1c9c18-fde9-4cc9-9686-0f7cc0972aaf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:21.807124 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9c1c9c18-fde9-4cc9-9686-0f7cc0972aaf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:21.807898 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9c1c9c18-fde9-4cc9-9686-0f7cc0972aaf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:21.808372 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9c1c9c18-fde9-4cc9-9686-0f7cc0972aaf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:21.808799 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9c1c9c18-fde9-4cc9-9686-0f7cc0972aaf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:22.631392 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-9c1c9c18-fde9-4cc9-9686-0f7cc0972aaf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:22.649993 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-9c1c9c18-fde9-4cc9-9686-0f7cc0972aaf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1292 microversion: 2.1 time: 0.894644 Oct 28 09:45:22.666216 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 18/39] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:21 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1292 bytes in 911 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:22.671630 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3611195c-0beb-43e7-a6d0-a0af96556da5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:22.672423 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-3611195c-0beb-43e7-a6d0-a0af96556da5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:22.679603 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3611195c-0beb-43e7-a6d0-a0af96556da5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:22.680321 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3611195c-0beb-43e7-a6d0-a0af96556da5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:22.680920 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3611195c-0beb-43e7-a6d0-a0af96556da5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:22.681771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3611195c-0beb-43e7-a6d0-a0af96556da5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:22.682635 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3611195c-0beb-43e7-a6d0-a0af96556da5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:22.683231 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3611195c-0beb-43e7-a6d0-a0af96556da5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:22.834492 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3611195c-0beb-43e7-a6d0-a0af96556da5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:22.846994 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3611195c-0beb-43e7-a6d0-a0af96556da5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:22.850277 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3611195c-0beb-43e7-a6d0-a0af96556da5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:22.852239 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-3611195c-0beb-43e7-a6d0-a0af96556da5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1333 microversion: 2.20 time: 0.184699 Oct 28 09:45:22.853017 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 19/40] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:22 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1333 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:22.860392 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-bd4f7030-7053-46be-83ea-d9e4f19d0eb2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:22.861275 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-bd4f7030-7053-46be-83ea-d9e4f19d0eb2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:22.869909 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bd4f7030-7053-46be-83ea-d9e4f19d0eb2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:22.870835 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bd4f7030-7053-46be-83ea-d9e4f19d0eb2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:22.871490 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bd4f7030-7053-46be-83ea-d9e4f19d0eb2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:22.872419 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bd4f7030-7053-46be-83ea-d9e4f19d0eb2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:22.873060 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bd4f7030-7053-46be-83ea-d9e4f19d0eb2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:22.873548 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bd4f7030-7053-46be-83ea-d9e4f19d0eb2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:23.029259 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-bd4f7030-7053-46be-83ea-d9e4f19d0eb2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:23.041661 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-bd4f7030-7053-46be-83ea-d9e4f19d0eb2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1245 microversion: 2.1 time: 0.186633 Oct 28 09:45:23.042504 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 20/41] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:22 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1245 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:23.050354 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b804e89a-ac2e-4078-95ca-802c936491f7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:23.051287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b804e89a-ac2e-4078-95ca-802c936491f7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:23.063631 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b804e89a-ac2e-4078-95ca-802c936491f7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:23.066342 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b804e89a-ac2e-4078-95ca-802c936491f7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:23.067216 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b804e89a-ac2e-4078-95ca-802c936491f7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:23.069179 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b804e89a-ac2e-4078-95ca-802c936491f7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:23.069869 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b804e89a-ac2e-4078-95ca-802c936491f7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:23.070582 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b804e89a-ac2e-4078-95ca-802c936491f7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:23.170718 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Flavor limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:45:23.171604 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Image limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:45:23.172265 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b804e89a-ac2e-4078-95ca-802c936491f7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:23.173224 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Flavor pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:45:23.174006 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Image pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:45:23.175305 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:45:23.184339 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b804e89a-ac2e-4078-95ca-802c936491f7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1210 microversion: 2.1 time: 0.140603 Oct 28 09:45:23.184930 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 21/42] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:23 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1210 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:23.190881 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:23.191364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:23.191785 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:23.192668 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:23.193279 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:23.193780 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:23.213089 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Getting quotas for project f2fcce993e6b4388bde493f0702fd49d. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:45:23.217212 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Getting quotas for user a4237ca76acd4f7483aaf66e1f5d2eea and project f2fcce993e6b4388bde493f0702fd49d. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:45:23.230033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Going to run 1 instances... {{(pid=127030) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:45:23.232432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127030) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:45:23.245010 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=11 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:23.290055 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c6ef998f-2e12-460e-855a-1b2754ed6ff1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 2.738708 Oct 28 09:45:23.294965 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 22/43] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:45:20 2022] POST /compute/v2.1/servers => generated 401 bytes in 2742 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 09:45:23.303831 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-587ca4c7-37f3-4e1c-ac7e-3d7ac299e0a8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:23.304450 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-587ca4c7-37f3-4e1c-ac7e-3d7ac299e0a8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:23.392819 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-587ca4c7-37f3-4e1c-ac7e-3d7ac299e0a8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:23.400974 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-587ca4c7-37f3-4e1c-ac7e-3d7ac299e0a8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1238 microversion: 2.1 time: 0.100876 Oct 28 09:45:23.401574 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 22/44] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:23 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1238 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:23.681712 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b343f096-de35-44f5-9dfb-0929d2e7e01a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:23.682756 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b343f096-de35-44f5-9dfb-0929d2e7e01a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:23.706653 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b343f096-de35-44f5-9dfb-0929d2e7e01a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:23.707139 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b343f096-de35-44f5-9dfb-0929d2e7e01a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:23.707578 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b343f096-de35-44f5-9dfb-0929d2e7e01a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:23.708279 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b343f096-de35-44f5-9dfb-0929d2e7e01a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:23.708727 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b343f096-de35-44f5-9dfb-0929d2e7e01a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:23.709180 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b343f096-de35-44f5-9dfb-0929d2e7e01a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:23.806745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b343f096-de35-44f5-9dfb-0929d2e7e01a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:23.839865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b343f096-de35-44f5-9dfb-0929d2e7e01a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1292 microversion: 2.1 time: 0.170676 Oct 28 09:45:23.850611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 23/45] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:23 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1292 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:23.894676 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-7d5eb011-e4b8-4938-b714-dce824bdd3e0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:23.895449 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-7d5eb011-e4b8-4938-b714-dce824bdd3e0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:23.922999 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7d5eb011-e4b8-4938-b714-dce824bdd3e0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:23.930245 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7d5eb011-e4b8-4938-b714-dce824bdd3e0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:23.931091 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7d5eb011-e4b8-4938-b714-dce824bdd3e0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:23.932150 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7d5eb011-e4b8-4938-b714-dce824bdd3e0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:23.932657 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7d5eb011-e4b8-4938-b714-dce824bdd3e0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:23.933187 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7d5eb011-e4b8-4938-b714-dce824bdd3e0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:24.068171 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2f511c26-0103-4e13-a155-c3815da030c8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:24.076726 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2f511c26-0103-4e13-a155-c3815da030c8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:24.086409 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2f511c26-0103-4e13-a155-c3815da030c8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:24.087040 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2f511c26-0103-4e13-a155-c3815da030c8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:24.087570 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2f511c26-0103-4e13-a155-c3815da030c8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:24.088600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2f511c26-0103-4e13-a155-c3815da030c8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:24.089307 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2f511c26-0103-4e13-a155-c3815da030c8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:24.089932 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2f511c26-0103-4e13-a155-c3815da030c8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:24.215063 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7d5eb011-e4b8-4938-b714-dce824bdd3e0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:24.254878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7d5eb011-e4b8-4938-b714-dce824bdd3e0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:24.266902 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7d5eb011-e4b8-4938-b714-dce824bdd3e0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:24.278209 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-7d5eb011-e4b8-4938-b714-dce824bdd3e0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1333 microversion: 2.20 time: 0.394835 Oct 28 09:45:24.294031 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 23/46] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:23 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1333 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:24.321880 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c3af88ad-ff6d-4e8d-ad4d-a17e9dc1a35e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:24.324379 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c3af88ad-ff6d-4e8d-ad4d-a17e9dc1a35e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:24.348116 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2f511c26-0103-4e13-a155-c3815da030c8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:24.348666 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c3af88ad-ff6d-4e8d-ad4d-a17e9dc1a35e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:24.348666 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c3af88ad-ff6d-4e8d-ad4d-a17e9dc1a35e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:24.349143 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c3af88ad-ff6d-4e8d-ad4d-a17e9dc1a35e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:24.350496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c3af88ad-ff6d-4e8d-ad4d-a17e9dc1a35e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:24.350496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c3af88ad-ff6d-4e8d-ad4d-a17e9dc1a35e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:24.350855 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c3af88ad-ff6d-4e8d-ad4d-a17e9dc1a35e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:24.391470 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2f511c26-0103-4e13-a155-c3815da030c8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1245 microversion: 2.1 time: 0.315886 Oct 28 09:45:24.394995 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 24/47] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:24 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1245 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:24.426377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8255bb1c-a7d2-4cd3-9c40-aee5ebc6cd11 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:24.426990 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8255bb1c-a7d2-4cd3-9c40-aee5ebc6cd11 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:24.471277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8255bb1c-a7d2-4cd3-9c40-aee5ebc6cd11 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:24.471768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8255bb1c-a7d2-4cd3-9c40-aee5ebc6cd11 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:24.483538 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8255bb1c-a7d2-4cd3-9c40-aee5ebc6cd11 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:24.486289 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8255bb1c-a7d2-4cd3-9c40-aee5ebc6cd11 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:24.487987 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8255bb1c-a7d2-4cd3-9c40-aee5ebc6cd11 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:24.489950 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8255bb1c-a7d2-4cd3-9c40-aee5ebc6cd11 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:24.507167 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c3af88ad-ff6d-4e8d-ad4d-a17e9dc1a35e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:24.520815 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c3af88ad-ff6d-4e8d-ad4d-a17e9dc1a35e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1210 microversion: 2.1 time: 0.223847 Oct 28 09:45:24.521414 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 24/48] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:24 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1210 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:24.640349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8255bb1c-a7d2-4cd3-9c40-aee5ebc6cd11 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:24.649004 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8255bb1c-a7d2-4cd3-9c40-aee5ebc6cd11 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1242 microversion: 2.1 time: 0.226393 Oct 28 09:45:24.649572 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 25/49] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:24 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1242 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:24.876249 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-f94ba087-6013-436a-ba44-7a1169d11287 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:24.877417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-f94ba087-6013-436a-ba44-7a1169d11287 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:24.892397 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f94ba087-6013-436a-ba44-7a1169d11287 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:24.893243 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f94ba087-6013-436a-ba44-7a1169d11287 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:24.893773 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f94ba087-6013-436a-ba44-7a1169d11287 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:24.894608 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f94ba087-6013-436a-ba44-7a1169d11287 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:24.895153 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f94ba087-6013-436a-ba44-7a1169d11287 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:24.895759 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f94ba087-6013-436a-ba44-7a1169d11287 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:25.067072 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-f94ba087-6013-436a-ba44-7a1169d11287 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:25.084793 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-f94ba087-6013-436a-ba44-7a1169d11287 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1280 microversion: 2.1 time: 0.209997 Oct 28 09:45:25.094648 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 25/50] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:24 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1280 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:25.315096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f1385e4d-93f7-436a-af6e-20692afe12ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:25.315711 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f1385e4d-93f7-436a-af6e-20692afe12ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:25.322786 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1385e4d-93f7-436a-af6e-20692afe12ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:25.327259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1385e4d-93f7-436a-af6e-20692afe12ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:25.327762 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1385e4d-93f7-436a-af6e-20692afe12ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:25.339096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1385e4d-93f7-436a-af6e-20692afe12ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:25.339596 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1385e4d-93f7-436a-af6e-20692afe12ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:25.340033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1385e4d-93f7-436a-af6e-20692afe12ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:25.422230 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-31afa583-1fdd-46c8-a09d-2ba4f01cfba1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:25.422878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-31afa583-1fdd-46c8-a09d-2ba4f01cfba1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:25.451288 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-31afa583-1fdd-46c8-a09d-2ba4f01cfba1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:25.452342 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-31afa583-1fdd-46c8-a09d-2ba4f01cfba1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:25.452962 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-31afa583-1fdd-46c8-a09d-2ba4f01cfba1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:25.460247 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-31afa583-1fdd-46c8-a09d-2ba4f01cfba1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:25.461938 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-31afa583-1fdd-46c8-a09d-2ba4f01cfba1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:25.464005 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-31afa583-1fdd-46c8-a09d-2ba4f01cfba1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:25.559562 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f1385e4d-93f7-436a-af6e-20692afe12ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:25.580084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f1385e4d-93f7-436a-af6e-20692afe12ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:25.588909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f1385e4d-93f7-436a-af6e-20692afe12ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:25.589956 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f1385e4d-93f7-436a-af6e-20692afe12ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1333 microversion: 2.20 time: 0.278964 Oct 28 09:45:25.610807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 26/51] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:25 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1333 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:25.637812 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-641ef4bb-9893-45a7-9e79-719536cbc8f8 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:25.643035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-641ef4bb-9893-45a7-9e79-719536cbc8f8 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:25.663425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-641ef4bb-9893-45a7-9e79-719536cbc8f8 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:25.668032 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-641ef4bb-9893-45a7-9e79-719536cbc8f8 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:25.668696 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-641ef4bb-9893-45a7-9e79-719536cbc8f8 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:25.670047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-641ef4bb-9893-45a7-9e79-719536cbc8f8 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:25.676005 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-641ef4bb-9893-45a7-9e79-719536cbc8f8 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:25.676965 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-641ef4bb-9893-45a7-9e79-719536cbc8f8 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:26.172879 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-31afa583-1fdd-46c8-a09d-2ba4f01cfba1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:26.183600 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-31afa583-1fdd-46c8-a09d-2ba4f01cfba1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1237 microversion: 2.1 time: 0.765149 Oct 28 09:45:26.202760 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 26/52] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:25 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1237 bytes in 785 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:26.207814 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-13bdaea1-f43f-4d7d-865a-f44ccd95619b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:26.221236 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-13bdaea1-f43f-4d7d-865a-f44ccd95619b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:26.245566 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-13bdaea1-f43f-4d7d-865a-f44ccd95619b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:26.246947 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-13bdaea1-f43f-4d7d-865a-f44ccd95619b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:26.250651 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-13bdaea1-f43f-4d7d-865a-f44ccd95619b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:26.254354 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-13bdaea1-f43f-4d7d-865a-f44ccd95619b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:26.255072 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-13bdaea1-f43f-4d7d-865a-f44ccd95619b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:26.255890 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-13bdaea1-f43f-4d7d-865a-f44ccd95619b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:26.384569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-641ef4bb-9893-45a7-9e79-719536cbc8f8 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:26.394839 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-641ef4bb-9893-45a7-9e79-719536cbc8f8 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1266 microversion: 2.1 time: 0.780864 Oct 28 09:45:26.422818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 27/53] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:25 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1266 bytes in 811 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:26.442054 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7134d607-3e02-49bc-aa93-a1a4bee132c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:26.449728 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7134d607-3e02-49bc-aa93-a1a4bee132c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:26.480293 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7134d607-3e02-49bc-aa93-a1a4bee132c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:26.481126 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7134d607-3e02-49bc-aa93-a1a4bee132c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:26.481838 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7134d607-3e02-49bc-aa93-a1a4bee132c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:26.482885 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7134d607-3e02-49bc-aa93-a1a4bee132c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:26.483547 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7134d607-3e02-49bc-aa93-a1a4bee132c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:26.484338 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7134d607-3e02-49bc-aa93-a1a4bee132c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:26.620739 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-13bdaea1-f43f-4d7d-865a-f44ccd95619b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:26.637502 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-13bdaea1-f43f-4d7d-865a-f44ccd95619b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1234 microversion: 2.1 time: 0.426412 Oct 28 09:45:26.641524 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 27/54] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:26 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1234 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:26.672831 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-97d815c5-393a-4036-8e50-2c4fc00e8f02 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:26.674625 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-97d815c5-393a-4036-8e50-2c4fc00e8f02 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:26.700595 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-97d815c5-393a-4036-8e50-2c4fc00e8f02 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:26.704310 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-97d815c5-393a-4036-8e50-2c4fc00e8f02 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:26.706323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-97d815c5-393a-4036-8e50-2c4fc00e8f02 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:26.708660 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-97d815c5-393a-4036-8e50-2c4fc00e8f02 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:26.713739 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-97d815c5-393a-4036-8e50-2c4fc00e8f02 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:26.714989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-97d815c5-393a-4036-8e50-2c4fc00e8f02 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:27.196205 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-97d815c5-393a-4036-8e50-2c4fc00e8f02 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:27.247972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-97d815c5-393a-4036-8e50-2c4fc00e8f02 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:27.249602 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-97d815c5-393a-4036-8e50-2c4fc00e8f02 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:27.268259 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-97d815c5-393a-4036-8e50-2c4fc00e8f02 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1339 microversion: 2.20 time: 0.608274 Oct 28 09:45:27.268976 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 28/55] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:26 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1339 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:27.274932 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-19276bb3-0026-4bec-9755-aa76b6b612ee tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:27.275866 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-19276bb3-0026-4bec-9755-aa76b6b612ee tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:27.307311 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=8 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:27.308728 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-19276bb3-0026-4bec-9755-aa76b6b612ee tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:27.309212 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-19276bb3-0026-4bec-9755-aa76b6b612ee tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:27.313372 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-19276bb3-0026-4bec-9755-aa76b6b612ee tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:27.321051 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-19276bb3-0026-4bec-9755-aa76b6b612ee tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:27.323920 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-19276bb3-0026-4bec-9755-aa76b6b612ee tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:27.324545 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-19276bb3-0026-4bec-9755-aa76b6b612ee tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:27.906711 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-19276bb3-0026-4bec-9755-aa76b6b612ee tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:27.917357 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-19276bb3-0026-4bec-9755-aa76b6b612ee tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1237 microversion: 2.1 time: 0.647266 Oct 28 09:45:27.922346 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 29/56] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:27 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1237 bytes in 653 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:27.934974 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3d437cac-fe88-4642-926d-0da00bca691e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:27.934974 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-3d437cac-fe88-4642-926d-0da00bca691e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:27.939223 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3d437cac-fe88-4642-926d-0da00bca691e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:27.939988 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3d437cac-fe88-4642-926d-0da00bca691e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:27.941827 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3d437cac-fe88-4642-926d-0da00bca691e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:27.942971 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3d437cac-fe88-4642-926d-0da00bca691e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:27.945590 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3d437cac-fe88-4642-926d-0da00bca691e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:27.946051 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3d437cac-fe88-4642-926d-0da00bca691e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:28.005709 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7134d607-3e02-49bc-aa93-a1a4bee132c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:28.016505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7134d607-3e02-49bc-aa93-a1a4bee132c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 1.589828 Oct 28 09:45:28.017676 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 28/57] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:26 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 1591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:28.024710 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a56a3d57-8ba5-4cc3-bdde-4c0e47906fc2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:28.026089 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a56a3d57-8ba5-4cc3-bdde-4c0e47906fc2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:28.042595 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a56a3d57-8ba5-4cc3-bdde-4c0e47906fc2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:28.043314 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a56a3d57-8ba5-4cc3-bdde-4c0e47906fc2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:28.045411 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a56a3d57-8ba5-4cc3-bdde-4c0e47906fc2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:28.046291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a56a3d57-8ba5-4cc3-bdde-4c0e47906fc2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:28.051645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a56a3d57-8ba5-4cc3-bdde-4c0e47906fc2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:28.052462 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a56a3d57-8ba5-4cc3-bdde-4c0e47906fc2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:28.079923 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3d437cac-fe88-4642-926d-0da00bca691e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:28.094163 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-3d437cac-fe88-4642-926d-0da00bca691e tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1266 microversion: 2.1 time: 0.168782 Oct 28 09:45:28.094793 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 30/58] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:27 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1266 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:28.159381 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a56a3d57-8ba5-4cc3-bdde-4c0e47906fc2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:28.169031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a56a3d57-8ba5-4cc3-bdde-4c0e47906fc2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1234 microversion: 2.1 time: 0.150097 Oct 28 09:45:28.170432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 29/59] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:28 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1234 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:28.304605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b47aa673-d9d9-40be-bedb-9092f2c6fa7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:28.305182 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b47aa673-d9d9-40be-bedb-9092f2c6fa7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:28.312203 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b47aa673-d9d9-40be-bedb-9092f2c6fa7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:28.313180 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b47aa673-d9d9-40be-bedb-9092f2c6fa7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:28.313830 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b47aa673-d9d9-40be-bedb-9092f2c6fa7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:28.314721 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b47aa673-d9d9-40be-bedb-9092f2c6fa7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:28.315315 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b47aa673-d9d9-40be-bedb-9092f2c6fa7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:28.315883 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b47aa673-d9d9-40be-bedb-9092f2c6fa7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:28.546320 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b47aa673-d9d9-40be-bedb-9092f2c6fa7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:28.570383 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b47aa673-d9d9-40be-bedb-9092f2c6fa7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:28.580724 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b47aa673-d9d9-40be-bedb-9092f2c6fa7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:28.584606 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b47aa673-d9d9-40be-bedb-9092f2c6fa7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1339 microversion: 2.20 time: 0.283385 Oct 28 09:45:28.585851 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 31/60] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:28 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1339 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:28.949793 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-531f2e80-4b97-4826-ab7f-8f1e67f04da2 req-d7417497-98ab-478f-a058-f61f51e78c9f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7", "tag": "a69a90a6-23e5-4976-90d8-d36b10ff6b35"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:28.954339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-2e28727f-6ded-4704-b634-9a73a7b908c0 req-7810b899-788c-486d-ad15-fd60cc757001 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3b7d7553-45b6-4a10-868e-b653d211245d", "tag": "bb15e303-a14b-4992-96a6-836072bb6949"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:28.971594 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-531f2e80-4b97-4826-ab7f-8f1e67f04da2 req-d7417497-98ab-478f-a058-f61f51e78c9f service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:28.972302 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-531f2e80-4b97-4826-ab7f-8f1e67f04da2 req-d7417497-98ab-478f-a058-f61f51e78c9f service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:28.972878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-531f2e80-4b97-4826-ab7f-8f1e67f04da2 req-d7417497-98ab-478f-a058-f61f51e78c9f service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:28.972999 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-2e28727f-6ded-4704-b634-9a73a7b908c0 req-7810b899-788c-486d-ad15-fd60cc757001 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:28.973446 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-2e28727f-6ded-4704-b634-9a73a7b908c0 req-7810b899-788c-486d-ad15-fd60cc757001 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:28.973864 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-2e28727f-6ded-4704-b634-9a73a7b908c0 req-7810b899-788c-486d-ad15-fd60cc757001 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:28.993959 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-531f2e80-4b97-4826-ab7f-8f1e67f04da2 req-d7417497-98ab-478f-a058-f61f51e78c9f service nova] Creating event network-changed:a69a90a6-23e5-4976-90d8-d36b10ff6b35 for instance cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 on hv-cinder-zed-3 Oct 28 09:45:28.994512 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-531f2e80-4b97-4826-ab7f-8f1e67f04da2 req-d7417497-98ab-478f-a058-f61f51e78c9f service nova] Acquiring lock "compute-rpcapi-router" {{(pid=127029) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 28 09:45:28.994899 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-531f2e80-4b97-4826-ab7f-8f1e67f04da2 req-d7417497-98ab-478f-a058-f61f51e78c9f service nova] Acquired lock "compute-rpcapi-router" {{(pid=127029) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 28 09:45:28.995603 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-2e28727f-6ded-4704-b634-9a73a7b908c0 req-7810b899-788c-486d-ad15-fd60cc757001 service nova] Creating event network-changed:bb15e303-a14b-4992-96a6-836072bb6949 for instance 3b7d7553-45b6-4a10-868e-b653d211245d on hv-cinder-zed-3 Oct 28 09:45:28.995922 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3ca6e47-1d0c-49b2-b124-5a43c5f70183 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:28.996118 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-2e28727f-6ded-4704-b634-9a73a7b908c0 req-7810b899-788c-486d-ad15-fd60cc757001 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=127030) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 28 09:45:28.996383 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3ca6e47-1d0c-49b2-b124-5a43c5f70183 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:28.996494 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-2e28727f-6ded-4704-b634-9a73a7b908c0 req-7810b899-788c-486d-ad15-fd60cc757001 service nova] Acquired lock "compute-rpcapi-router" {{(pid=127030) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 28 09:45:28.996800 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3ca6e47-1d0c-49b2-b124-5a43c5f70183 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:28.997485 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d399ec56-b202-4e4d-8b13-0cf16353507a None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:28.997607 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3ca6e47-1d0c-49b2-b124-5a43c5f70183 None None] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:28.997928 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d399ec56-b202-4e4d-8b13-0cf16353507a None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:28.998037 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3ca6e47-1d0c-49b2-b124-5a43c5f70183 None None] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:28.998338 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3ca6e47-1d0c-49b2-b124-5a43c5f70183 None None] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:28.998441 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d399ec56-b202-4e4d-8b13-0cf16353507a None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:28.999056 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d399ec56-b202-4e4d-8b13-0cf16353507a None None] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:28.999487 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d399ec56-b202-4e4d-8b13-0cf16353507a None None] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:28.999893 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d399ec56-b202-4e4d-8b13-0cf16353507a None None] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:29.008509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.compute.rpcapi [None req-d399ec56-b202-4e4d-8b13-0cf16353507a None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 28 09:45:29.009389 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d399ec56-b202-4e4d-8b13-0cf16353507a None None] Releasing lock "compute-rpcapi-router" {{(pid=127030) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 28 09:45:29.012140 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.compute.rpcapi [None req-d3ca6e47-1d0c-49b2-b124-5a43c5f70183 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 28 09:45:29.013144 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3ca6e47-1d0c-49b2-b124-5a43c5f70183 None None] Releasing lock "compute-rpcapi-router" {{(pid=127029) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 28 09:45:29.025035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d399ec56-b202-4e4d-8b13-0cf16353507a None None] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.150288 Oct 28 09:45:29.025800 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 30/61] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:45:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:45:29.031054 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7b8607ea-d702-4f32-a923-7951ff645e88 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:29.031171 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d3ca6e47-1d0c-49b2-b124-5a43c5f70183 None None] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.146402 Oct 28 09:45:29.031732 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 32/62] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:45:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:45:29.031836 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7b8607ea-d702-4f32-a923-7951ff645e88 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:29.041141 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d76949be-e9a3-431d-adc5-67e2a6c2435b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:29.041843 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d76949be-e9a3-431d-adc5-67e2a6c2435b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:29.042039 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7b8607ea-d702-4f32-a923-7951ff645e88 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:29.042552 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7b8607ea-d702-4f32-a923-7951ff645e88 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:29.042984 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7b8607ea-d702-4f32-a923-7951ff645e88 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:29.043670 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7b8607ea-d702-4f32-a923-7951ff645e88 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:29.044109 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7b8607ea-d702-4f32-a923-7951ff645e88 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:29.044524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7b8607ea-d702-4f32-a923-7951ff645e88 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:29.047935 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d76949be-e9a3-431d-adc5-67e2a6c2435b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:29.048508 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d76949be-e9a3-431d-adc5-67e2a6c2435b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:29.049017 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d76949be-e9a3-431d-adc5-67e2a6c2435b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:29.049793 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d76949be-e9a3-431d-adc5-67e2a6c2435b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:29.050295 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d76949be-e9a3-431d-adc5-67e2a6c2435b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:29.050817 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d76949be-e9a3-431d-adc5-67e2a6c2435b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:29.055184 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:29.063659 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=100 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:29.125558 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7b8607ea-d702-4f32-a923-7951ff645e88 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:29.133778 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7b8607ea-d702-4f32-a923-7951ff645e88 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1237 microversion: 2.1 time: 0.106576 Oct 28 09:45:29.134348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 31/63] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:29 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1237 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:29.138988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-141d08ee-30c8-4cc0-ae38-da9068548bee tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:29.139581 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-141d08ee-30c8-4cc0-ae38-da9068548bee tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:29.145695 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-141d08ee-30c8-4cc0-ae38-da9068548bee tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:29.146142 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-141d08ee-30c8-4cc0-ae38-da9068548bee tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:29.147211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-141d08ee-30c8-4cc0-ae38-da9068548bee tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:29.147828 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-141d08ee-30c8-4cc0-ae38-da9068548bee tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:29.148394 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-141d08ee-30c8-4cc0-ae38-da9068548bee tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:29.148870 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-141d08ee-30c8-4cc0-ae38-da9068548bee tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:29.226611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-141d08ee-30c8-4cc0-ae38-da9068548bee tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:29.235669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-141d08ee-30c8-4cc0-ae38-da9068548bee tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1266 microversion: 2.1 time: 0.100070 Oct 28 09:45:29.236524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 32/64] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:29 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1266 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:29.241380 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9d485a0f-fa82-400c-9028-db2af28329d7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:29.241981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9d485a0f-fa82-400c-9028-db2af28329d7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:29.247953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9d485a0f-fa82-400c-9028-db2af28329d7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:29.248736 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9d485a0f-fa82-400c-9028-db2af28329d7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:29.249328 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9d485a0f-fa82-400c-9028-db2af28329d7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:29.250178 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9d485a0f-fa82-400c-9028-db2af28329d7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:29.250805 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9d485a0f-fa82-400c-9028-db2af28329d7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:29.251992 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9d485a0f-fa82-400c-9028-db2af28329d7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:29.260472 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d76949be-e9a3-431d-adc5-67e2a6c2435b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:29.268971 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d76949be-e9a3-431d-adc5-67e2a6c2435b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.232554 Oct 28 09:45:29.269741 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 33/65] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:29 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:29.322089 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-9d485a0f-fa82-400c-9028-db2af28329d7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:29.330154 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9d485a0f-fa82-400c-9028-db2af28329d7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1234 microversion: 2.1 time: 0.092776 Oct 28 09:45:29.330752 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 33/66] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:29 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1234 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:29.611006 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-9e05ff79-ad22-4d57-bce9-2791640b0ef0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:29.611614 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-9e05ff79-ad22-4d57-bce9-2791640b0ef0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:29.617793 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9e05ff79-ad22-4d57-bce9-2791640b0ef0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:29.618210 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9e05ff79-ad22-4d57-bce9-2791640b0ef0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:29.618716 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9e05ff79-ad22-4d57-bce9-2791640b0ef0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:29.619327 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9e05ff79-ad22-4d57-bce9-2791640b0ef0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:29.619880 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9e05ff79-ad22-4d57-bce9-2791640b0ef0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:29.620274 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9e05ff79-ad22-4d57-bce9-2791640b0ef0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:29.685011 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-9e05ff79-ad22-4d57-bce9-2791640b0ef0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:29.694232 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-9e05ff79-ad22-4d57-bce9-2791640b0ef0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:29.695736 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-9e05ff79-ad22-4d57-bce9-2791640b0ef0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:29.696864 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-9e05ff79-ad22-4d57-bce9-2791640b0ef0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1339 microversion: 2.20 time: 0.089773 Oct 28 09:45:29.697415 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 34/67] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:29 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1339 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:30.158563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4d5d4964-9756-4aea-938d-d851c40c41cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:30.159171 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4d5d4964-9756-4aea-938d-d851c40c41cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:30.165543 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4d5d4964-9756-4aea-938d-d851c40c41cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:30.166276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4d5d4964-9756-4aea-938d-d851c40c41cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:30.166728 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4d5d4964-9756-4aea-938d-d851c40c41cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:30.167415 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4d5d4964-9756-4aea-938d-d851c40c41cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:30.167905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4d5d4964-9756-4aea-938d-d851c40c41cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:30.168317 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4d5d4964-9756-4aea-938d-d851c40c41cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:30.230119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-4d5d4964-9756-4aea-938d-d851c40c41cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:30.237938 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4d5d4964-9756-4aea-938d-d851c40c41cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1237 microversion: 2.1 time: 0.084262 Oct 28 09:45:30.238515 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 34/68] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:30 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1237 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:30.259253 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5c3df734-565c-449a-afbb-cd2ac7b6a3c4 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:30.259872 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5c3df734-565c-449a-afbb-cd2ac7b6a3c4 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:30.279028 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5c3df734-565c-449a-afbb-cd2ac7b6a3c4 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:30.279936 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5c3df734-565c-449a-afbb-cd2ac7b6a3c4 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:30.280021 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5c3df734-565c-449a-afbb-cd2ac7b6a3c4 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:30.280715 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5c3df734-565c-449a-afbb-cd2ac7b6a3c4 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:30.281166 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5c3df734-565c-449a-afbb-cd2ac7b6a3c4 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:30.281583 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5c3df734-565c-449a-afbb-cd2ac7b6a3c4 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:30.309747 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-94badc7e-ec76-4eab-9164-e18b580e6c95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:30.327406 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-94badc7e-ec76-4eab-9164-e18b580e6c95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:30.343549 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-94badc7e-ec76-4eab-9164-e18b580e6c95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:30.344397 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-94badc7e-ec76-4eab-9164-e18b580e6c95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:30.345175 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-94badc7e-ec76-4eab-9164-e18b580e6c95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:30.346077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-94badc7e-ec76-4eab-9164-e18b580e6c95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:30.346736 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-94badc7e-ec76-4eab-9164-e18b580e6c95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:30.347329 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-94badc7e-ec76-4eab-9164-e18b580e6c95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:30.417627 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5c3df734-565c-449a-afbb-cd2ac7b6a3c4 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:30.425999 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5c3df734-565c-449a-afbb-cd2ac7b6a3c4 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1266 microversion: 2.1 time: 0.171167 Oct 28 09:45:30.432156 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 35/69] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:30 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1266 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:30.438637 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-483a373c-39a7-4c00-931c-171c59e56d9e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:30.440105 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-483a373c-39a7-4c00-931c-171c59e56d9e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:30.447434 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-483a373c-39a7-4c00-931c-171c59e56d9e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:30.448115 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-483a373c-39a7-4c00-931c-171c59e56d9e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:30.448690 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-483a373c-39a7-4c00-931c-171c59e56d9e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:30.449719 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-483a373c-39a7-4c00-931c-171c59e56d9e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:30.450297 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-483a373c-39a7-4c00-931c-171c59e56d9e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:30.450866 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-483a373c-39a7-4c00-931c-171c59e56d9e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:30.560786 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-483a373c-39a7-4c00-931c-171c59e56d9e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:30.569126 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-483a373c-39a7-4c00-931c-171c59e56d9e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1234 microversion: 2.1 time: 0.135698 Oct 28 09:45:30.583491 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 36/70] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:30 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1234 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:30.620213 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-94badc7e-ec76-4eab-9164-e18b580e6c95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:30.627946 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-94badc7e-ec76-4eab-9164-e18b580e6c95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.330604 Oct 28 09:45:30.628513 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 35/71] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:30 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:30.722586 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-94666427-b5bd-45a3-a96b-8d3b8fbedd5c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:30.723234 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-94666427-b5bd-45a3-a96b-8d3b8fbedd5c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:30.729994 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94666427-b5bd-45a3-a96b-8d3b8fbedd5c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:30.730537 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94666427-b5bd-45a3-a96b-8d3b8fbedd5c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:30.730970 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94666427-b5bd-45a3-a96b-8d3b8fbedd5c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:30.731657 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94666427-b5bd-45a3-a96b-8d3b8fbedd5c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:30.732099 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94666427-b5bd-45a3-a96b-8d3b8fbedd5c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:30.732585 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94666427-b5bd-45a3-a96b-8d3b8fbedd5c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:30.813855 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-94666427-b5bd-45a3-a96b-8d3b8fbedd5c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:30.826052 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-94666427-b5bd-45a3-a96b-8d3b8fbedd5c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:30.827555 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-94666427-b5bd-45a3-a96b-8d3b8fbedd5c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:30.828736 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-94666427-b5bd-45a3-a96b-8d3b8fbedd5c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1339 microversion: 2.20 time: 0.110155 Oct 28 09:45:30.829475 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 37/72] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:30 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1339 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:31.264169 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0bf3bf37-d3ec-4822-a306-dbbf9d1d1fe4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:31.264778 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0bf3bf37-d3ec-4822-a306-dbbf9d1d1fe4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:31.279440 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bf3bf37-d3ec-4822-a306-dbbf9d1d1fe4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:31.284513 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bf3bf37-d3ec-4822-a306-dbbf9d1d1fe4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:31.284948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bf3bf37-d3ec-4822-a306-dbbf9d1d1fe4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:31.285678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bf3bf37-d3ec-4822-a306-dbbf9d1d1fe4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:31.286128 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bf3bf37-d3ec-4822-a306-dbbf9d1d1fe4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:31.286579 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bf3bf37-d3ec-4822-a306-dbbf9d1d1fe4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:31.418623 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0bf3bf37-d3ec-4822-a306-dbbf9d1d1fe4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:31.430152 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0bf3bf37-d3ec-4822-a306-dbbf9d1d1fe4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1237 microversion: 2.1 time: 0.170518 Oct 28 09:45:31.431400 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 36/73] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:31 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1237 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:31.471830 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-798434c4-c391-4500-9423-b42a189027d6 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:31.477885 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-798434c4-c391-4500-9423-b42a189027d6 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:31.495167 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-798434c4-c391-4500-9423-b42a189027d6 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:31.496156 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-798434c4-c391-4500-9423-b42a189027d6 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:31.497230 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-798434c4-c391-4500-9423-b42a189027d6 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:31.498549 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-798434c4-c391-4500-9423-b42a189027d6 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:31.499490 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-798434c4-c391-4500-9423-b42a189027d6 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:31.500052 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-798434c4-c391-4500-9423-b42a189027d6 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:31.593428 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-78c6a3d0-8a45-4b8e-aa7d-a39ab5568fd9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:31.594348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-78c6a3d0-8a45-4b8e-aa7d-a39ab5568fd9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:31.600957 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78c6a3d0-8a45-4b8e-aa7d-a39ab5568fd9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:31.601598 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78c6a3d0-8a45-4b8e-aa7d-a39ab5568fd9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:31.603117 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78c6a3d0-8a45-4b8e-aa7d-a39ab5568fd9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:31.603117 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78c6a3d0-8a45-4b8e-aa7d-a39ab5568fd9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:31.603599 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78c6a3d0-8a45-4b8e-aa7d-a39ab5568fd9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:31.604110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78c6a3d0-8a45-4b8e-aa7d-a39ab5568fd9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:31.656732 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-798434c4-c391-4500-9423-b42a189027d6 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:31.668613 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-798434c4-c391-4500-9423-b42a189027d6 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1274 microversion: 2.1 time: 0.203668 Oct 28 09:45:31.670425 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 38/74] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:31 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1274 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:31.680598 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b209f107-49fc-4318-9086-591fd0a29257 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:31.681667 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b209f107-49fc-4318-9086-591fd0a29257 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:31.717895 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b209f107-49fc-4318-9086-591fd0a29257 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:31.719159 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b209f107-49fc-4318-9086-591fd0a29257 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:31.719894 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b209f107-49fc-4318-9086-591fd0a29257 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:31.721362 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b209f107-49fc-4318-9086-591fd0a29257 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:31.725495 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b209f107-49fc-4318-9086-591fd0a29257 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:31.726670 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b209f107-49fc-4318-9086-591fd0a29257 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:31.768656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-78c6a3d0-8a45-4b8e-aa7d-a39ab5568fd9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:31.796842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-78c6a3d0-8a45-4b8e-aa7d-a39ab5568fd9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1234 microversion: 2.1 time: 0.207578 Oct 28 09:45:31.797430 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 37/75] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:31 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1234 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:31.856408 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3187fe03-eac7-41e1-8cf2-2db692bd0fc0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:31.858123 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3187fe03-eac7-41e1-8cf2-2db692bd0fc0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:31.873174 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3187fe03-eac7-41e1-8cf2-2db692bd0fc0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:31.873852 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3187fe03-eac7-41e1-8cf2-2db692bd0fc0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:31.874845 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3187fe03-eac7-41e1-8cf2-2db692bd0fc0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:31.877279 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3187fe03-eac7-41e1-8cf2-2db692bd0fc0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:31.878310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3187fe03-eac7-41e1-8cf2-2db692bd0fc0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:31.878940 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3187fe03-eac7-41e1-8cf2-2db692bd0fc0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:31.958253 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b209f107-49fc-4318-9086-591fd0a29257 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:31.977733 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3187fe03-eac7-41e1-8cf2-2db692bd0fc0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:31.983241 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b209f107-49fc-4318-9086-591fd0a29257 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.311202 Oct 28 09:45:31.998639 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3187fe03-eac7-41e1-8cf2-2db692bd0fc0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:31.998876 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 39/76] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:31 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:31.998996 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3187fe03-eac7-41e1-8cf2-2db692bd0fc0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:31.999261 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3187fe03-eac7-41e1-8cf2-2db692bd0fc0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1339 microversion: 2.20 time: 0.146998 Oct 28 09:45:32.007256 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 38/77] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:31 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1339 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:45:32.455062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-73f9e1bd-8c5e-43d5-8492-9447484d9611 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:32.455673 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-73f9e1bd-8c5e-43d5-8492-9447484d9611 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:32.461747 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73f9e1bd-8c5e-43d5-8492-9447484d9611 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:32.462558 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73f9e1bd-8c5e-43d5-8492-9447484d9611 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:32.463229 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73f9e1bd-8c5e-43d5-8492-9447484d9611 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:32.464281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73f9e1bd-8c5e-43d5-8492-9447484d9611 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:32.470799 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73f9e1bd-8c5e-43d5-8492-9447484d9611 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:32.471407 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73f9e1bd-8c5e-43d5-8492-9447484d9611 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:32.543703 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-73f9e1bd-8c5e-43d5-8492-9447484d9611 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:32.558814 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-73f9e1bd-8c5e-43d5-8492-9447484d9611 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1237 microversion: 2.1 time: 0.102573 Oct 28 09:45:32.560065 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 40/78] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:32 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1237 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:32.707548 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f92389cd-0359-4617-8e40-a9a00f1f8a90 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:32.707548 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f92389cd-0359-4617-8e40-a9a00f1f8a90 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:32.716807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f92389cd-0359-4617-8e40-a9a00f1f8a90 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:32.723420 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f92389cd-0359-4617-8e40-a9a00f1f8a90 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:32.724083 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f92389cd-0359-4617-8e40-a9a00f1f8a90 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:32.726016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f92389cd-0359-4617-8e40-a9a00f1f8a90 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:32.726568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f92389cd-0359-4617-8e40-a9a00f1f8a90 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:32.726942 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f92389cd-0359-4617-8e40-a9a00f1f8a90 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:32.821670 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f92389cd-0359-4617-8e40-a9a00f1f8a90 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:32.825230 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-a502de63-e990-4e97-bd84-40408caa6aac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:32.826566 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-a502de63-e990-4e97-bd84-40408caa6aac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:32.836527 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a502de63-e990-4e97-bd84-40408caa6aac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:32.837516 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f92389cd-0359-4617-8e40-a9a00f1f8a90 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1274 microversion: 2.1 time: 0.136245 Oct 28 09:45:32.838263 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a502de63-e990-4e97-bd84-40408caa6aac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:32.838991 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 39/79] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:32 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1274 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:32.839699 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a502de63-e990-4e97-bd84-40408caa6aac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:32.841256 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a502de63-e990-4e97-bd84-40408caa6aac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:32.844096 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a502de63-e990-4e97-bd84-40408caa6aac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:32.844789 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a502de63-e990-4e97-bd84-40408caa6aac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:32.935638 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-a502de63-e990-4e97-bd84-40408caa6aac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:32.944564 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-a502de63-e990-4e97-bd84-40408caa6aac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1234 microversion: 2.1 time: 0.123178 Oct 28 09:45:32.945686 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 41/80] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:32 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1234 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:33.009178 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f2a1b6f0-ef16-48ef-99f8-611ab27647af tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:33.009791 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f2a1b6f0-ef16-48ef-99f8-611ab27647af tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:33.020302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f2a1b6f0-ef16-48ef-99f8-611ab27647af tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:33.020808 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f2a1b6f0-ef16-48ef-99f8-611ab27647af tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:33.021270 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f2a1b6f0-ef16-48ef-99f8-611ab27647af tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:33.022493 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f2a1b6f0-ef16-48ef-99f8-611ab27647af tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:33.022942 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f2a1b6f0-ef16-48ef-99f8-611ab27647af tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:33.023355 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f2a1b6f0-ef16-48ef-99f8-611ab27647af tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:33.027762 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-76ac5ea0-1b8f-4402-bc92-21dcf1dc3ac9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:33.028751 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-76ac5ea0-1b8f-4402-bc92-21dcf1dc3ac9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:33.037111 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-76ac5ea0-1b8f-4402-bc92-21dcf1dc3ac9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:33.041894 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-76ac5ea0-1b8f-4402-bc92-21dcf1dc3ac9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:33.042886 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-76ac5ea0-1b8f-4402-bc92-21dcf1dc3ac9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:33.044219 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-76ac5ea0-1b8f-4402-bc92-21dcf1dc3ac9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:33.046320 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-76ac5ea0-1b8f-4402-bc92-21dcf1dc3ac9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:33.046865 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-76ac5ea0-1b8f-4402-bc92-21dcf1dc3ac9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:33.148644 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-76ac5ea0-1b8f-4402-bc92-21dcf1dc3ac9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:33.161311 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-76ac5ea0-1b8f-4402-bc92-21dcf1dc3ac9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:33.170682 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-76ac5ea0-1b8f-4402-bc92-21dcf1dc3ac9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:33.172010 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-76ac5ea0-1b8f-4402-bc92-21dcf1dc3ac9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1339 microversion: 2.20 time: 0.150200 Oct 28 09:45:33.172750 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 42/81] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:33 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1339 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:33.225924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=32 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:33.317446 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f2a1b6f0-ef16-48ef-99f8-611ab27647af tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:33.329590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f2a1b6f0-ef16-48ef-99f8-611ab27647af tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.324285 Oct 28 09:45:33.330292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 40/82] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:33 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:33.582893 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-47db5725-dd83-4b2f-bd29-9f8ee8f98f86 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:33.583579 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-47db5725-dd83-4b2f-bd29-9f8ee8f98f86 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:33.585626 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=28 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:33.592203 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-47db5725-dd83-4b2f-bd29-9f8ee8f98f86 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:33.592955 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-47db5725-dd83-4b2f-bd29-9f8ee8f98f86 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:33.593714 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-47db5725-dd83-4b2f-bd29-9f8ee8f98f86 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:33.594450 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-47db5725-dd83-4b2f-bd29-9f8ee8f98f86 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:33.594908 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-47db5725-dd83-4b2f-bd29-9f8ee8f98f86 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:33.595568 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-47db5725-dd83-4b2f-bd29-9f8ee8f98f86 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:33.666337 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-47db5725-dd83-4b2f-bd29-9f8ee8f98f86 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:33.674456 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-47db5725-dd83-4b2f-bd29-9f8ee8f98f86 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1237 microversion: 2.1 time: 0.095338 Oct 28 09:45:33.675275 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 43/83] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:33 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1237 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:33.867501 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-297d9368-9c2f-4629-b8b8-317f4ff0fede tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:33.868758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-297d9368-9c2f-4629-b8b8-317f4ff0fede tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:33.874701 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-297d9368-9c2f-4629-b8b8-317f4ff0fede tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:33.875296 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-297d9368-9c2f-4629-b8b8-317f4ff0fede tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:33.875780 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-297d9368-9c2f-4629-b8b8-317f4ff0fede tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:33.876440 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-297d9368-9c2f-4629-b8b8-317f4ff0fede tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:33.876885 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-297d9368-9c2f-4629-b8b8-317f4ff0fede tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:33.877303 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-297d9368-9c2f-4629-b8b8-317f4ff0fede tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:33.968728 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-297d9368-9c2f-4629-b8b8-317f4ff0fede tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:33.978691 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-52170de9-59b5-4ba6-8a84-2296d19952b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:33.979965 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-297d9368-9c2f-4629-b8b8-317f4ff0fede tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1274 microversion: 2.1 time: 0.116720 Oct 28 09:45:33.980105 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-52170de9-59b5-4ba6-8a84-2296d19952b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:33.980543 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 41/84] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:33 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1274 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:33.996374 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-52170de9-59b5-4ba6-8a84-2296d19952b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:33.996892 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-52170de9-59b5-4ba6-8a84-2296d19952b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:33.997576 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-52170de9-59b5-4ba6-8a84-2296d19952b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:33.999127 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-52170de9-59b5-4ba6-8a84-2296d19952b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:33.999664 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-52170de9-59b5-4ba6-8a84-2296d19952b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:34.000220 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-52170de9-59b5-4ba6-8a84-2296d19952b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:34.108741 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-52170de9-59b5-4ba6-8a84-2296d19952b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:34.117299 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-52170de9-59b5-4ba6-8a84-2296d19952b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1234 microversion: 2.1 time: 0.143643 Oct 28 09:45:34.117887 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 44/85] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:33 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1234 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:34.200475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f6d599d1-2e30-4dc6-821d-f40220b00ac4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:34.201105 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f6d599d1-2e30-4dc6-821d-f40220b00ac4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:34.207046 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6d599d1-2e30-4dc6-821d-f40220b00ac4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:34.207517 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6d599d1-2e30-4dc6-821d-f40220b00ac4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:34.207941 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6d599d1-2e30-4dc6-821d-f40220b00ac4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:34.208615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6d599d1-2e30-4dc6-821d-f40220b00ac4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:34.209617 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6d599d1-2e30-4dc6-821d-f40220b00ac4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:34.210032 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6d599d1-2e30-4dc6-821d-f40220b00ac4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:34.291988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f6d599d1-2e30-4dc6-821d-f40220b00ac4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:34.301030 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f6d599d1-2e30-4dc6-821d-f40220b00ac4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:34.302886 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f6d599d1-2e30-4dc6-821d-f40220b00ac4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:34.303998 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f6d599d1-2e30-4dc6-821d-f40220b00ac4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1339 microversion: 2.20 time: 0.107922 Oct 28 09:45:34.305039 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 42/86] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:34 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1339 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:34.352387 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-ffacd19f-07cf-4b7c-aaf8-786fba9f1c4c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:34.353141 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-ffacd19f-07cf-4b7c-aaf8-786fba9f1c4c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:34.359426 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ffacd19f-07cf-4b7c-aaf8-786fba9f1c4c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:34.360158 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ffacd19f-07cf-4b7c-aaf8-786fba9f1c4c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:34.361080 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ffacd19f-07cf-4b7c-aaf8-786fba9f1c4c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:34.361952 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ffacd19f-07cf-4b7c-aaf8-786fba9f1c4c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:34.362847 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ffacd19f-07cf-4b7c-aaf8-786fba9f1c4c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:34.364323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ffacd19f-07cf-4b7c-aaf8-786fba9f1c4c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:34.524315 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-ffacd19f-07cf-4b7c-aaf8-786fba9f1c4c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:34.533090 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-ffacd19f-07cf-4b7c-aaf8-786fba9f1c4c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.184721 Oct 28 09:45:34.533671 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 45/87] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:34 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:34.700591 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-bef3480c-5789-4abf-9e2a-0d2f7563ce4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:34.701382 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-bef3480c-5789-4abf-9e2a-0d2f7563ce4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:34.712799 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bef3480c-5789-4abf-9e2a-0d2f7563ce4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:34.713416 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bef3480c-5789-4abf-9e2a-0d2f7563ce4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:34.713947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bef3480c-5789-4abf-9e2a-0d2f7563ce4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:34.714911 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bef3480c-5789-4abf-9e2a-0d2f7563ce4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:34.715651 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bef3480c-5789-4abf-9e2a-0d2f7563ce4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:34.716308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bef3480c-5789-4abf-9e2a-0d2f7563ce4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:34.789281 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-bef3480c-5789-4abf-9e2a-0d2f7563ce4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:34.799531 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-bef3480c-5789-4abf-9e2a-0d2f7563ce4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1237 microversion: 2.1 time: 0.104675 Oct 28 09:45:34.806672 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 43/88] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:34 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1237 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:35.004568 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5e7da074-d0f1-4335-ae83-7063818932cd tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:35.005161 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5e7da074-d0f1-4335-ae83-7063818932cd tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:35.012984 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e7da074-d0f1-4335-ae83-7063818932cd tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:35.013591 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e7da074-d0f1-4335-ae83-7063818932cd tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:35.014085 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e7da074-d0f1-4335-ae83-7063818932cd tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:35.014879 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e7da074-d0f1-4335-ae83-7063818932cd tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:35.015435 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e7da074-d0f1-4335-ae83-7063818932cd tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:35.016043 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e7da074-d0f1-4335-ae83-7063818932cd tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:35.083317 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5e7da074-d0f1-4335-ae83-7063818932cd tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:35.091220 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5e7da074-d0f1-4335-ae83-7063818932cd tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1274 microversion: 2.1 time: 0.090502 Oct 28 09:45:35.091822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 46/89] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:34 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1274 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:35.140613 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c6686e79-58ad-4279-8a1b-f2ee6c44ca51 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:35.141191 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c6686e79-58ad-4279-8a1b-f2ee6c44ca51 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:35.147945 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c6686e79-58ad-4279-8a1b-f2ee6c44ca51 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:35.148709 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c6686e79-58ad-4279-8a1b-f2ee6c44ca51 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:35.149146 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c6686e79-58ad-4279-8a1b-f2ee6c44ca51 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:35.149839 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c6686e79-58ad-4279-8a1b-f2ee6c44ca51 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:35.150284 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c6686e79-58ad-4279-8a1b-f2ee6c44ca51 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:35.150736 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c6686e79-58ad-4279-8a1b-f2ee6c44ca51 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:35.229348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c6686e79-58ad-4279-8a1b-f2ee6c44ca51 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:35.239124 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c6686e79-58ad-4279-8a1b-f2ee6c44ca51 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1234 microversion: 2.1 time: 0.102321 Oct 28 09:45:35.239697 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 44/90] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:35 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1234 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:35.329421 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-fc66a278-fc76-46b6-8add-fe85874bb808 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:35.330257 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-fc66a278-fc76-46b6-8add-fe85874bb808 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:35.336738 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fc66a278-fc76-46b6-8add-fe85874bb808 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:35.337572 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fc66a278-fc76-46b6-8add-fe85874bb808 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:35.337999 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fc66a278-fc76-46b6-8add-fe85874bb808 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:35.338885 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fc66a278-fc76-46b6-8add-fe85874bb808 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:35.339343 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fc66a278-fc76-46b6-8add-fe85874bb808 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:35.339781 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fc66a278-fc76-46b6-8add-fe85874bb808 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:35.401549 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-fc66a278-fc76-46b6-8add-fe85874bb808 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:35.410744 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-fc66a278-fc76-46b6-8add-fe85874bb808 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:35.412277 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-fc66a278-fc76-46b6-8add-fe85874bb808 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:35.413417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-fc66a278-fc76-46b6-8add-fe85874bb808 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1339 microversion: 2.20 time: 0.087960 Oct 28 09:45:35.423926 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 47/91] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:35 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1339 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:35.557439 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-67e6e697-1f2d-4ad5-843f-79277c666d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:35.559640 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-67e6e697-1f2d-4ad5-843f-79277c666d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:35.566016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-67e6e697-1f2d-4ad5-843f-79277c666d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:35.566532 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-67e6e697-1f2d-4ad5-843f-79277c666d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:35.566961 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-67e6e697-1f2d-4ad5-843f-79277c666d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:35.567658 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-67e6e697-1f2d-4ad5-843f-79277c666d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:35.568296 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-67e6e697-1f2d-4ad5-843f-79277c666d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:35.569003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-67e6e697-1f2d-4ad5-843f-79277c666d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:35.708171 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-67e6e697-1f2d-4ad5-843f-79277c666d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:35.716819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-67e6e697-1f2d-4ad5-843f-79277c666d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.163833 Oct 28 09:45:35.717821 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 45/92] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:35 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:35.824148 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-1a8c9ab0-90c4-4636-83f1-266ce67c598d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:35.824788 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-1a8c9ab0-90c4-4636-83f1-266ce67c598d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:35.830916 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1a8c9ab0-90c4-4636-83f1-266ce67c598d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:35.831826 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1a8c9ab0-90c4-4636-83f1-266ce67c598d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:35.832263 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1a8c9ab0-90c4-4636-83f1-266ce67c598d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:35.832954 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1a8c9ab0-90c4-4636-83f1-266ce67c598d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:35.833674 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1a8c9ab0-90c4-4636-83f1-266ce67c598d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:35.833843 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1a8c9ab0-90c4-4636-83f1-266ce67c598d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:35.907589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-1a8c9ab0-90c4-4636-83f1-266ce67c598d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:35.916507 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-1a8c9ab0-90c4-4636-83f1-266ce67c598d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1237 microversion: 2.1 time: 0.096317 Oct 28 09:45:35.918291 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 48/93] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:35 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1237 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:36.114154 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b8cef13c-22ba-45b9-ab7d-77581575b653 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:36.114794 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b8cef13c-22ba-45b9-ab7d-77581575b653 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:36.121575 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b8cef13c-22ba-45b9-ab7d-77581575b653 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:36.122055 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b8cef13c-22ba-45b9-ab7d-77581575b653 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:36.122523 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b8cef13c-22ba-45b9-ab7d-77581575b653 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:36.123206 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b8cef13c-22ba-45b9-ab7d-77581575b653 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:36.123656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b8cef13c-22ba-45b9-ab7d-77581575b653 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:36.124076 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b8cef13c-22ba-45b9-ab7d-77581575b653 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:36.185677 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b8cef13c-22ba-45b9-ab7d-77581575b653 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:36.193924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b8cef13c-22ba-45b9-ab7d-77581575b653 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1274 microversion: 2.1 time: 0.083001 Oct 28 09:45:36.194508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 46/94] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:36 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1274 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:36.262886 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-62d941cd-bd7b-45a1-a063-bb2d8f81575d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:36.263492 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-62d941cd-bd7b-45a1-a063-bb2d8f81575d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:36.269343 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-62d941cd-bd7b-45a1-a063-bb2d8f81575d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:36.269829 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-62d941cd-bd7b-45a1-a063-bb2d8f81575d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:36.270256 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-62d941cd-bd7b-45a1-a063-bb2d8f81575d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:36.270965 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-62d941cd-bd7b-45a1-a063-bb2d8f81575d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:36.271414 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-62d941cd-bd7b-45a1-a063-bb2d8f81575d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:36.271834 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-62d941cd-bd7b-45a1-a063-bb2d8f81575d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:36.333051 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-62d941cd-bd7b-45a1-a063-bb2d8f81575d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:36.336520 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:36.341286 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-62d941cd-bd7b-45a1-a063-bb2d8f81575d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1234 microversion: 2.1 time: 0.082158 Oct 28 09:45:36.341860 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 49/95] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:36 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1234 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:36.439678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d29509d3-ffaa-4e24-ae97-b61e61aff170 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:36.440172 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d29509d3-ffaa-4e24-ae97-b61e61aff170 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:36.446677 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d29509d3-ffaa-4e24-ae97-b61e61aff170 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:36.447218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d29509d3-ffaa-4e24-ae97-b61e61aff170 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:36.447692 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d29509d3-ffaa-4e24-ae97-b61e61aff170 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:36.448392 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d29509d3-ffaa-4e24-ae97-b61e61aff170 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:36.448954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d29509d3-ffaa-4e24-ae97-b61e61aff170 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:36.449470 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d29509d3-ffaa-4e24-ae97-b61e61aff170 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:36.511011 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d29509d3-ffaa-4e24-ae97-b61e61aff170 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:36.518962 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d29509d3-ffaa-4e24-ae97-b61e61aff170 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:36.520394 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d29509d3-ffaa-4e24-ae97-b61e61aff170 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:36.522037 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d29509d3-ffaa-4e24-ae97-b61e61aff170 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1339 microversion: 2.20 time: 0.085738 Oct 28 09:45:36.522615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 47/96] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:36 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1339 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:36.739702 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d00c8899-3336-4a6f-b25d-c0419ce739d2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:36.740367 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d00c8899-3336-4a6f-b25d-c0419ce739d2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:36.746369 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d00c8899-3336-4a6f-b25d-c0419ce739d2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:36.746883 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d00c8899-3336-4a6f-b25d-c0419ce739d2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:36.747316 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d00c8899-3336-4a6f-b25d-c0419ce739d2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:36.747996 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d00c8899-3336-4a6f-b25d-c0419ce739d2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:36.748446 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d00c8899-3336-4a6f-b25d-c0419ce739d2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:36.748865 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d00c8899-3336-4a6f-b25d-c0419ce739d2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:36.884226 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d00c8899-3336-4a6f-b25d-c0419ce739d2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:36.892666 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d00c8899-3336-4a6f-b25d-c0419ce739d2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.156944 Oct 28 09:45:36.893339 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 50/97] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:36 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:36.941057 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d85d4e53-f24a-4f9b-9977-75c1bb40b516 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:36.941911 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d85d4e53-f24a-4f9b-9977-75c1bb40b516 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:36.948175 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d85d4e53-f24a-4f9b-9977-75c1bb40b516 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:36.948661 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d85d4e53-f24a-4f9b-9977-75c1bb40b516 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:36.949237 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d85d4e53-f24a-4f9b-9977-75c1bb40b516 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:36.949826 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d85d4e53-f24a-4f9b-9977-75c1bb40b516 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:36.950217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d85d4e53-f24a-4f9b-9977-75c1bb40b516 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:36.950661 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d85d4e53-f24a-4f9b-9977-75c1bb40b516 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:36.954907 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=100 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:37.026237 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d85d4e53-f24a-4f9b-9977-75c1bb40b516 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:37.039850 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d85d4e53-f24a-4f9b-9977-75c1bb40b516 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1293 microversion: 2.1 time: 0.099176 Oct 28 09:45:37.040487 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 48/98] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:36 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1293 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:37.216895 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-90cda2ff-27c4-4198-9da6-5b1fc9a1f478 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:37.217612 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-90cda2ff-27c4-4198-9da6-5b1fc9a1f478 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:37.227792 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-90cda2ff-27c4-4198-9da6-5b1fc9a1f478 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:37.228866 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-90cda2ff-27c4-4198-9da6-5b1fc9a1f478 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:37.228866 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-90cda2ff-27c4-4198-9da6-5b1fc9a1f478 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:37.229519 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-90cda2ff-27c4-4198-9da6-5b1fc9a1f478 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:37.229980 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-90cda2ff-27c4-4198-9da6-5b1fc9a1f478 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:37.230745 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-90cda2ff-27c4-4198-9da6-5b1fc9a1f478 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:37.365575 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-048d8a21-e6fd-4c3b-bf02-34011f38e593 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:37.366041 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-048d8a21-e6fd-4c3b-bf02-34011f38e593 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:37.372930 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-048d8a21-e6fd-4c3b-bf02-34011f38e593 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:37.375167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-048d8a21-e6fd-4c3b-bf02-34011f38e593 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:37.375776 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-048d8a21-e6fd-4c3b-bf02-34011f38e593 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:37.376652 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-048d8a21-e6fd-4c3b-bf02-34011f38e593 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:37.377255 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-048d8a21-e6fd-4c3b-bf02-34011f38e593 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:37.378060 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-048d8a21-e6fd-4c3b-bf02-34011f38e593 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:37.389232 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-90cda2ff-27c4-4198-9da6-5b1fc9a1f478 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:37.400195 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-90cda2ff-27c4-4198-9da6-5b1fc9a1f478 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1272 microversion: 2.1 time: 0.187556 Oct 28 09:45:37.400773 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 51/99] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:37 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1272 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:37.547524 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-a326c3d8-53b2-4f38-ab93-8fed1a6e1433 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:37.548157 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-a326c3d8-53b2-4f38-ab93-8fed1a6e1433 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:37.561377 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a326c3d8-53b2-4f38-ab93-8fed1a6e1433 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:37.561968 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a326c3d8-53b2-4f38-ab93-8fed1a6e1433 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:37.562528 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a326c3d8-53b2-4f38-ab93-8fed1a6e1433 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:37.563316 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a326c3d8-53b2-4f38-ab93-8fed1a6e1433 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:37.564385 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-048d8a21-e6fd-4c3b-bf02-34011f38e593 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:37.572359 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a326c3d8-53b2-4f38-ab93-8fed1a6e1433 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:37.573165 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a326c3d8-53b2-4f38-ab93-8fed1a6e1433 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:37.580113 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-048d8a21-e6fd-4c3b-bf02-34011f38e593 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1234 microversion: 2.1 time: 0.219511 Oct 28 09:45:37.580889 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 49/100] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:37 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1234 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:37.666532 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-a326c3d8-53b2-4f38-ab93-8fed1a6e1433 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:37.678803 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-a326c3d8-53b2-4f38-ab93-8fed1a6e1433 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:37.680917 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-a326c3d8-53b2-4f38-ab93-8fed1a6e1433 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:37.684241 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-a326c3d8-53b2-4f38-ab93-8fed1a6e1433 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1339 microversion: 2.20 time: 0.140915 Oct 28 09:45:37.684777 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 52/101] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:37 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1339 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:45:37.918512 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e737d1be-1292-4757-9020-c422e211a75b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:37.919315 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e737d1be-1292-4757-9020-c422e211a75b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:37.942430 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e737d1be-1292-4757-9020-c422e211a75b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:37.948028 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e737d1be-1292-4757-9020-c422e211a75b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:37.952357 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e737d1be-1292-4757-9020-c422e211a75b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:37.953140 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e737d1be-1292-4757-9020-c422e211a75b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:37.953625 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e737d1be-1292-4757-9020-c422e211a75b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:37.969688 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e737d1be-1292-4757-9020-c422e211a75b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:38.066775 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-054db7d6-01f2-410c-9673-ed47277853e9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:38.067562 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-054db7d6-01f2-410c-9673-ed47277853e9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:38.078113 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-054db7d6-01f2-410c-9673-ed47277853e9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:38.082139 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-054db7d6-01f2-410c-9673-ed47277853e9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:38.082139 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-054db7d6-01f2-410c-9673-ed47277853e9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:38.092473 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-054db7d6-01f2-410c-9673-ed47277853e9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:38.095362 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-054db7d6-01f2-410c-9673-ed47277853e9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:38.098139 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-054db7d6-01f2-410c-9673-ed47277853e9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:38.645877 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-054db7d6-01f2-410c-9673-ed47277853e9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:38.662429 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-054db7d6-01f2-410c-9673-ed47277853e9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1301 microversion: 2.1 time: 0.599268 Oct 28 09:45:38.665212 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 53/102] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:38 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1301 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:38.690288 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-ea500589-c646-474f-9c3f-42544f349688 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:38.692298 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-ea500589-c646-474f-9c3f-42544f349688 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:38.705350 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ea500589-c646-474f-9c3f-42544f349688 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:38.705638 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ea500589-c646-474f-9c3f-42544f349688 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:38.708117 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ea500589-c646-474f-9c3f-42544f349688 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:38.709125 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ea500589-c646-474f-9c3f-42544f349688 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:38.709646 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ea500589-c646-474f-9c3f-42544f349688 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:38.710299 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ea500589-c646-474f-9c3f-42544f349688 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:38.910018 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e737d1be-1292-4757-9020-c422e211a75b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:38.919078 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e737d1be-1292-4757-9020-c422e211a75b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 1.007121 Oct 28 09:45:38.922503 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 50/103] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:37 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 1011 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:38.931632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-fccf4de8-8c8c-46da-bf6d-6d26ad11dbfe tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:38.932739 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-fccf4de8-8c8c-46da-bf6d-6d26ad11dbfe tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:38.950776 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fccf4de8-8c8c-46da-bf6d-6d26ad11dbfe tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:38.954278 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fccf4de8-8c8c-46da-bf6d-6d26ad11dbfe tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:38.955023 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fccf4de8-8c8c-46da-bf6d-6d26ad11dbfe tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:38.956564 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fccf4de8-8c8c-46da-bf6d-6d26ad11dbfe tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:38.957506 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fccf4de8-8c8c-46da-bf6d-6d26ad11dbfe tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:38.958236 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fccf4de8-8c8c-46da-bf6d-6d26ad11dbfe tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:39.007589 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=10 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:39.011126 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=6 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:39.091835 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-fccf4de8-8c8c-46da-bf6d-6d26ad11dbfe tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:39.102487 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-fccf4de8-8c8c-46da-bf6d-6d26ad11dbfe tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1290 microversion: 2.1 time: 0.178781 Oct 28 09:45:39.107833 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 51/104] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:38 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1290 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:39.120326 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-61850f08-c4a6-4b14-bdba-7e5b7200ef2b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:39.138192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-61850f08-c4a6-4b14-bdba-7e5b7200ef2b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:39.150205 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-61850f08-c4a6-4b14-bdba-7e5b7200ef2b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:39.150605 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-61850f08-c4a6-4b14-bdba-7e5b7200ef2b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:39.151030 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-61850f08-c4a6-4b14-bdba-7e5b7200ef2b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:39.152002 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-61850f08-c4a6-4b14-bdba-7e5b7200ef2b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:39.152443 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-61850f08-c4a6-4b14-bdba-7e5b7200ef2b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:39.154035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-61850f08-c4a6-4b14-bdba-7e5b7200ef2b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:39.177983 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-ea500589-c646-474f-9c3f-42544f349688 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:39.223613 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-ea500589-c646-474f-9c3f-42544f349688 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1314 microversion: 2.1 time: 0.557084 Oct 28 09:45:39.224685 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 54/105] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:38 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1314 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:39.690030 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3817b9e9-5fbe-428f-bc47-361f96e7cadd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:39.691311 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-3817b9e9-5fbe-428f-bc47-361f96e7cadd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:39.707709 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3817b9e9-5fbe-428f-bc47-361f96e7cadd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:39.709256 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3817b9e9-5fbe-428f-bc47-361f96e7cadd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:39.709887 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3817b9e9-5fbe-428f-bc47-361f96e7cadd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:39.710779 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3817b9e9-5fbe-428f-bc47-361f96e7cadd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:39.711654 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3817b9e9-5fbe-428f-bc47-361f96e7cadd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:39.712228 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3817b9e9-5fbe-428f-bc47-361f96e7cadd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:39.975922 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3817b9e9-5fbe-428f-bc47-361f96e7cadd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:39.984475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-61850f08-c4a6-4b14-bdba-7e5b7200ef2b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:39.985979 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-3817b9e9-5fbe-428f-bc47-361f96e7cadd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1299 microversion: 2.1 time: 0.301435 Oct 28 09:45:39.986625 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 55/106] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:39 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1299 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:40.001507 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-61850f08-c4a6-4b14-bdba-7e5b7200ef2b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:40.003400 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3c7ceb6d-131c-4807-9909-571734de6873 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:40.004356 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-3c7ceb6d-131c-4807-9909-571734de6873 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:40.005982 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-61850f08-c4a6-4b14-bdba-7e5b7200ef2b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:40.009002 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-61850f08-c4a6-4b14-bdba-7e5b7200ef2b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.899950 Oct 28 09:45:40.011621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 52/107] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:39 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 903 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:40.016066 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3c7ceb6d-131c-4807-9909-571734de6873 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:40.016654 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3c7ceb6d-131c-4807-9909-571734de6873 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:40.017093 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3c7ceb6d-131c-4807-9909-571734de6873 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:40.017891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3c7ceb6d-131c-4807-9909-571734de6873 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:40.018482 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3c7ceb6d-131c-4807-9909-571734de6873 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:40.019389 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3c7ceb6d-131c-4807-9909-571734de6873 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:40.125140 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-49b29f3f-e633-4871-b14d-432a6cb9a5c1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:40.125944 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-49b29f3f-e633-4871-b14d-432a6cb9a5c1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:40.133167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-49b29f3f-e633-4871-b14d-432a6cb9a5c1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:40.133730 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-49b29f3f-e633-4871-b14d-432a6cb9a5c1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:40.134531 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-49b29f3f-e633-4871-b14d-432a6cb9a5c1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:40.135281 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-49b29f3f-e633-4871-b14d-432a6cb9a5c1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:40.138566 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-49b29f3f-e633-4871-b14d-432a6cb9a5c1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:40.139090 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-49b29f3f-e633-4871-b14d-432a6cb9a5c1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:40.285140 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-49b29f3f-e633-4871-b14d-432a6cb9a5c1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:40.293683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-49b29f3f-e633-4871-b14d-432a6cb9a5c1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1296 microversion: 2.1 time: 0.172210 Oct 28 09:45:40.294755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 53/108] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:40 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1296 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:40.305416 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7e802e29-7bf1-41d1-8b60-dce1925d6cf0 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:40.307449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7e802e29-7bf1-41d1-8b60-dce1925d6cf0 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:40.313684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e802e29-7bf1-41d1-8b60-dce1925d6cf0 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:40.314274 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e802e29-7bf1-41d1-8b60-dce1925d6cf0 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:40.315013 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e802e29-7bf1-41d1-8b60-dce1925d6cf0 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:40.315920 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e802e29-7bf1-41d1-8b60-dce1925d6cf0 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:40.316547 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e802e29-7bf1-41d1-8b60-dce1925d6cf0 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:40.317281 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e802e29-7bf1-41d1-8b60-dce1925d6cf0 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:40.467392 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3c7ceb6d-131c-4807-9909-571734de6873 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:40.481149 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-3c7ceb6d-131c-4807-9909-571734de6873 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.488001 Oct 28 09:45:40.481761 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 56/109] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:39 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:40.503124 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-7942a633-00cb-4544-8e35-f9e9f27300f2 req-19e4ab2f-b7d2-4ede-bd95-57c2798078a0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "088d0e80-3838-4ada-938b-f26f194d5abc", "tag": "a903c483-3505-4c5d-9652-f34609414d9d"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:40.528129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-7942a633-00cb-4544-8e35-f9e9f27300f2 req-19e4ab2f-b7d2-4ede-bd95-57c2798078a0 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:40.528615 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-7942a633-00cb-4544-8e35-f9e9f27300f2 req-19e4ab2f-b7d2-4ede-bd95-57c2798078a0 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:40.529038 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-7942a633-00cb-4544-8e35-f9e9f27300f2 req-19e4ab2f-b7d2-4ede-bd95-57c2798078a0 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:40.548286 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-7942a633-00cb-4544-8e35-f9e9f27300f2 req-19e4ab2f-b7d2-4ede-bd95-57c2798078a0 service nova] Creating event network-changed:a903c483-3505-4c5d-9652-f34609414d9d for instance 088d0e80-3838-4ada-938b-f26f194d5abc on hv-cinder-zed-3 Oct 28 09:45:40.554144 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-7942a633-00cb-4544-8e35-f9e9f27300f2 req-19e4ab2f-b7d2-4ede-bd95-57c2798078a0 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058275 Oct 28 09:45:40.555106 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 57/110] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:45:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:45:40.894120 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7e802e29-7bf1-41d1-8b60-dce1925d6cf0 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:40.902628 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7e802e29-7bf1-41d1-8b60-dce1925d6cf0 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1314 microversion: 2.1 time: 0.604540 Oct 28 09:45:40.918845 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 54/111] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:40 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1314 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:41.011281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-64177475-f65c-4276-accc-3bffc4659154 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:41.012606 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-64177475-f65c-4276-accc-3bffc4659154 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:41.019472 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-64177475-f65c-4276-accc-3bffc4659154 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:41.020620 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-64177475-f65c-4276-accc-3bffc4659154 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:41.021204 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-64177475-f65c-4276-accc-3bffc4659154 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:41.021784 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-64177475-f65c-4276-accc-3bffc4659154 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:41.022231 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-64177475-f65c-4276-accc-3bffc4659154 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:41.022856 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-64177475-f65c-4276-accc-3bffc4659154 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:41.046226 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ab9cf82e-fffd-4ed1-8de7-d47274d78f78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:41.047132 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ab9cf82e-fffd-4ed1-8de7-d47274d78f78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:41.059298 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab9cf82e-fffd-4ed1-8de7-d47274d78f78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:41.060379 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab9cf82e-fffd-4ed1-8de7-d47274d78f78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:41.060957 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab9cf82e-fffd-4ed1-8de7-d47274d78f78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:41.061847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab9cf82e-fffd-4ed1-8de7-d47274d78f78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:41.062720 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab9cf82e-fffd-4ed1-8de7-d47274d78f78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:41.063292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab9cf82e-fffd-4ed1-8de7-d47274d78f78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:41.263103 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-64177475-f65c-4276-accc-3bffc4659154 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:41.274747 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-64177475-f65c-4276-accc-3bffc4659154 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1299 microversion: 2.1 time: 0.268953 Oct 28 09:45:41.277277 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 58/112] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:41 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1299 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:41.327448 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6af194a9-19bb-49a0-96cb-e83333278d29 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:41.330686 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6af194a9-19bb-49a0-96cb-e83333278d29 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:41.351596 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6af194a9-19bb-49a0-96cb-e83333278d29 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:41.353513 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6af194a9-19bb-49a0-96cb-e83333278d29 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:41.354627 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6af194a9-19bb-49a0-96cb-e83333278d29 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:41.357464 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6af194a9-19bb-49a0-96cb-e83333278d29 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:41.358493 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6af194a9-19bb-49a0-96cb-e83333278d29 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:41.360002 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6af194a9-19bb-49a0-96cb-e83333278d29 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:41.613163 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6af194a9-19bb-49a0-96cb-e83333278d29 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:41.621366 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6af194a9-19bb-49a0-96cb-e83333278d29 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1296 microversion: 2.1 time: 0.304129 Oct 28 09:45:41.621955 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 59/113] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:41 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1296 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:41.650010 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-136cb664-41b9-4372-98f8-9d8153a05abc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:41.650660 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-136cb664-41b9-4372-98f8-9d8153a05abc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:41.673576 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-136cb664-41b9-4372-98f8-9d8153a05abc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:41.677255 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-136cb664-41b9-4372-98f8-9d8153a05abc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:41.678913 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-136cb664-41b9-4372-98f8-9d8153a05abc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:41.681589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-136cb664-41b9-4372-98f8-9d8153a05abc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:41.687804 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-136cb664-41b9-4372-98f8-9d8153a05abc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:41.688599 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-136cb664-41b9-4372-98f8-9d8153a05abc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:41.782150 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ab9cf82e-fffd-4ed1-8de7-d47274d78f78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:41.800561 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ab9cf82e-fffd-4ed1-8de7-d47274d78f78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:41.802739 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ab9cf82e-fffd-4ed1-8de7-d47274d78f78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:41.804074 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ab9cf82e-fffd-4ed1-8de7-d47274d78f78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.763443 Oct 28 09:45:41.820720 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 55/114] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:41 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 780 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:41.932065 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5dfc4c3d-9633-49e1-9f37-73d7d3256a95 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:41.934638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5dfc4c3d-9633-49e1-9f37-73d7d3256a95 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:41.948876 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5dfc4c3d-9633-49e1-9f37-73d7d3256a95 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:41.949540 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5dfc4c3d-9633-49e1-9f37-73d7d3256a95 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:41.950183 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5dfc4c3d-9633-49e1-9f37-73d7d3256a95 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:41.952197 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5dfc4c3d-9633-49e1-9f37-73d7d3256a95 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:41.953769 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5dfc4c3d-9633-49e1-9f37-73d7d3256a95 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:41.954876 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5dfc4c3d-9633-49e1-9f37-73d7d3256a95 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:42.368661 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-136cb664-41b9-4372-98f8-9d8153a05abc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:42.382786 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-136cb664-41b9-4372-98f8-9d8153a05abc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.747521 Oct 28 09:45:42.383402 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 60/115] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:41 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 761 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:42.411177 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c4ee4ba2-9c23-44b3-a611-d4324643a1ab tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:42.413292 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c4ee4ba2-9c23-44b3-a611-d4324643a1ab tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:42.426280 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c4ee4ba2-9c23-44b3-a611-d4324643a1ab tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:42.426990 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c4ee4ba2-9c23-44b3-a611-d4324643a1ab tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:42.428440 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c4ee4ba2-9c23-44b3-a611-d4324643a1ab tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:42.430320 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c4ee4ba2-9c23-44b3-a611-d4324643a1ab tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:42.431649 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c4ee4ba2-9c23-44b3-a611-d4324643a1ab tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:42.432891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c4ee4ba2-9c23-44b3-a611-d4324643a1ab tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:42.473270 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5dfc4c3d-9633-49e1-9f37-73d7d3256a95 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:42.483189 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5dfc4c3d-9633-49e1-9f37-73d7d3256a95 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1314 microversion: 2.1 time: 0.559437 Oct 28 09:45:42.487117 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 56/116] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:41 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1314 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:42.594319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-4e0849bd-393e-44b2-8358-d0152b148514 req-ab211ecd-b444-4653-a42f-a0cc06a79e06 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c716d5fd-f257-4f6a-8f52-18ad59e5c31e", "tag": "c2310b44-82bb-408f-85b5-8a5556b92e25"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:42.616536 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-4e0849bd-393e-44b2-8358-d0152b148514 req-ab211ecd-b444-4653-a42f-a0cc06a79e06 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:42.617014 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-4e0849bd-393e-44b2-8358-d0152b148514 req-ab211ecd-b444-4653-a42f-a0cc06a79e06 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:42.617627 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-4e0849bd-393e-44b2-8358-d0152b148514 req-ab211ecd-b444-4653-a42f-a0cc06a79e06 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:42.633519 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-4e0849bd-393e-44b2-8358-d0152b148514 req-ab211ecd-b444-4653-a42f-a0cc06a79e06 service nova] Creating event network-changed:c2310b44-82bb-408f-85b5-8a5556b92e25 for instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e on hv-cinder-zed-3 Oct 28 09:45:42.648267 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-4e0849bd-393e-44b2-8358-d0152b148514 req-ab211ecd-b444-4653-a42f-a0cc06a79e06 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057664 Oct 28 09:45:42.648880 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 57/117] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:45:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:45:42.654060 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3c1bca2e-8e4d-4b43-a417-64690b67a959 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:42.663011 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3c1bca2e-8e4d-4b43-a417-64690b67a959 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:42.680641 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c1bca2e-8e4d-4b43-a417-64690b67a959 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:42.681162 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c1bca2e-8e4d-4b43-a417-64690b67a959 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:42.681618 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c1bca2e-8e4d-4b43-a417-64690b67a959 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:42.682526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c1bca2e-8e4d-4b43-a417-64690b67a959 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:42.683149 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c1bca2e-8e4d-4b43-a417-64690b67a959 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:42.683569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c1bca2e-8e4d-4b43-a417-64690b67a959 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:42.855947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3c1bca2e-8e4d-4b43-a417-64690b67a959 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:42.866270 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3c1bca2e-8e4d-4b43-a417-64690b67a959 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1296 microversion: 2.1 time: 0.216171 Oct 28 09:45:42.866883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 58/118] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:42 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1296 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:42.878798 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-94077b49-50f6-48ca-a9af-d9986a5259a4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:42.883209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-94077b49-50f6-48ca-a9af-d9986a5259a4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:42.891692 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-94077b49-50f6-48ca-a9af-d9986a5259a4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:42.892406 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-94077b49-50f6-48ca-a9af-d9986a5259a4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:42.893014 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-94077b49-50f6-48ca-a9af-d9986a5259a4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:42.894051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-94077b49-50f6-48ca-a9af-d9986a5259a4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:42.894674 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-94077b49-50f6-48ca-a9af-d9986a5259a4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:42.895368 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-94077b49-50f6-48ca-a9af-d9986a5259a4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:42.944098 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c4ee4ba2-9c23-44b3-a611-d4324643a1ab tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:42.966722 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c4ee4ba2-9c23-44b3-a611-d4324643a1ab tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.560698 Oct 28 09:45:42.970330 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 61/119] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:42 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:43.416373 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5772bf9d-e55b-4dab-96db-4b1ac01d2733 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:43.417845 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5772bf9d-e55b-4dab-96db-4b1ac01d2733 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:43.435233 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5772bf9d-e55b-4dab-96db-4b1ac01d2733 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:43.437934 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5772bf9d-e55b-4dab-96db-4b1ac01d2733 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:43.438388 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5772bf9d-e55b-4dab-96db-4b1ac01d2733 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:43.439233 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5772bf9d-e55b-4dab-96db-4b1ac01d2733 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:43.440087 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5772bf9d-e55b-4dab-96db-4b1ac01d2733 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:43.443274 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-94077b49-50f6-48ca-a9af-d9986a5259a4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:43.446822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5772bf9d-e55b-4dab-96db-4b1ac01d2733 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:43.455654 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-94077b49-50f6-48ca-a9af-d9986a5259a4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:43.457187 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-94077b49-50f6-48ca-a9af-d9986a5259a4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:43.459584 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-94077b49-50f6-48ca-a9af-d9986a5259a4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.585572 Oct 28 09:45:43.459947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 59/120] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:42 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 587 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:43.513233 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-39ca7e45-a7e7-45bc-baa3-79c71db8df88 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:43.514659 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-39ca7e45-a7e7-45bc-baa3-79c71db8df88 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:43.521838 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39ca7e45-a7e7-45bc-baa3-79c71db8df88 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:43.522799 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39ca7e45-a7e7-45bc-baa3-79c71db8df88 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:43.523219 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39ca7e45-a7e7-45bc-baa3-79c71db8df88 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:43.524241 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39ca7e45-a7e7-45bc-baa3-79c71db8df88 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:43.524814 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39ca7e45-a7e7-45bc-baa3-79c71db8df88 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:43.525528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39ca7e45-a7e7-45bc-baa3-79c71db8df88 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:44.187696 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5772bf9d-e55b-4dab-96db-4b1ac01d2733 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:44.196310 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5772bf9d-e55b-4dab-96db-4b1ac01d2733 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.789057 Oct 28 09:45:44.210248 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 62/121] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:43 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 800 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:44.213077 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5d92e0d8-6a9f-4d82-b5df-9c09a4a6c343 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:44.214629 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5d92e0d8-6a9f-4d82-b5df-9c09a4a6c343 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:44.226617 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5d92e0d8-6a9f-4d82-b5df-9c09a4a6c343 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:44.226995 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5d92e0d8-6a9f-4d82-b5df-9c09a4a6c343 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:44.227503 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5d92e0d8-6a9f-4d82-b5df-9c09a4a6c343 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:44.228107 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5d92e0d8-6a9f-4d82-b5df-9c09a4a6c343 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:44.228665 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5d92e0d8-6a9f-4d82-b5df-9c09a4a6c343 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:44.228955 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5d92e0d8-6a9f-4d82-b5df-9c09a4a6c343 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:44.383105 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-39ca7e45-a7e7-45bc-baa3-79c71db8df88 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:44.394921 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-39ca7e45-a7e7-45bc-baa3-79c71db8df88 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1314 microversion: 2.1 time: 0.885607 Oct 28 09:45:44.397947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 60/122] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:43 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1314 bytes in 890 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:44.423625 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ce3a11a8-4acf-476f-9fbe-6249206a7036 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:44.427139 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ce3a11a8-4acf-476f-9fbe-6249206a7036 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:44.443943 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ce3a11a8-4acf-476f-9fbe-6249206a7036 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:44.444448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ce3a11a8-4acf-476f-9fbe-6249206a7036 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:44.445107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ce3a11a8-4acf-476f-9fbe-6249206a7036 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:44.458292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ce3a11a8-4acf-476f-9fbe-6249206a7036 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:44.460351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ce3a11a8-4acf-476f-9fbe-6249206a7036 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:44.463784 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ce3a11a8-4acf-476f-9fbe-6249206a7036 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:44.663802 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5d92e0d8-6a9f-4d82-b5df-9c09a4a6c343 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:44.677994 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5d92e0d8-6a9f-4d82-b5df-9c09a4a6c343 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.470049 Oct 28 09:45:44.690423 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 63/123] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:44 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:44.690423 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-800ae5e4-af82-4bba-a16d-68d54c79368c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:44.690423 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-800ae5e4-af82-4bba-a16d-68d54c79368c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:44.699555 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-800ae5e4-af82-4bba-a16d-68d54c79368c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:44.700657 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-800ae5e4-af82-4bba-a16d-68d54c79368c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:44.701507 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-800ae5e4-af82-4bba-a16d-68d54c79368c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:44.702533 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-800ae5e4-af82-4bba-a16d-68d54c79368c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:44.703181 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-800ae5e4-af82-4bba-a16d-68d54c79368c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:44.703977 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-800ae5e4-af82-4bba-a16d-68d54c79368c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:45.027243 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-800ae5e4-af82-4bba-a16d-68d54c79368c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:45.041623 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-800ae5e4-af82-4bba-a16d-68d54c79368c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:45.043981 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-800ae5e4-af82-4bba-a16d-68d54c79368c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:45.047385 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-800ae5e4-af82-4bba-a16d-68d54c79368c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.367020 Oct 28 09:45:45.047976 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 64/124] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:44 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:45.135970 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-10f00211-dfa7-49f9-aca6-7ab3376ce214 req-39f999ce-b6e0-4726-94a1-e7dbeb7f7e7b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b7d7553-45b6-4a10-868e-b653d211245d", "name": "network-vif-plugged", "status": "completed", "tag": "bb15e303-a14b-4992-96a6-836072bb6949"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:45.149831 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-10f00211-dfa7-49f9-aca6-7ab3376ce214 req-39f999ce-b6e0-4726-94a1-e7dbeb7f7e7b service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:45.150315 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-10f00211-dfa7-49f9-aca6-7ab3376ce214 req-39f999ce-b6e0-4726-94a1-e7dbeb7f7e7b service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:45.150817 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-10f00211-dfa7-49f9-aca6-7ab3376ce214 req-39f999ce-b6e0-4726-94a1-e7dbeb7f7e7b service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:45.167456 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-10f00211-dfa7-49f9-aca6-7ab3376ce214 req-39f999ce-b6e0-4726-94a1-e7dbeb7f7e7b service nova] Creating event network-vif-plugged:bb15e303-a14b-4992-96a6-836072bb6949 for instance 3b7d7553-45b6-4a10-868e-b653d211245d on hv-cinder-zed-3 Oct 28 09:45:45.173496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-10f00211-dfa7-49f9-aca6-7ab3376ce214 req-39f999ce-b6e0-4726-94a1-e7dbeb7f7e7b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.110998 Oct 28 09:45:45.174112 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 65/125] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:45:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:45:45.227700 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-a0cf07ec-976b-4162-b053-e9fddd43b73b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:45.228706 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-a0cf07ec-976b-4162-b053-e9fddd43b73b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:45.236851 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a0cf07ec-976b-4162-b053-e9fddd43b73b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:45.237535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ce3a11a8-4acf-476f-9fbe-6249206a7036 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:45.238976 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a0cf07ec-976b-4162-b053-e9fddd43b73b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:45.239614 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a0cf07ec-976b-4162-b053-e9fddd43b73b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:45.240831 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a0cf07ec-976b-4162-b053-e9fddd43b73b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:45.241510 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a0cf07ec-976b-4162-b053-e9fddd43b73b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:45.242707 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a0cf07ec-976b-4162-b053-e9fddd43b73b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:45.247471 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ce3a11a8-4acf-476f-9fbe-6249206a7036 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.848098 Oct 28 09:45:45.248721 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 61/126] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:44 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 850 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:45.431071 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-21712e1b-9cd0-4515-abb8-c3a93344b80f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:45.431779 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-21712e1b-9cd0-4515-abb8-c3a93344b80f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:45.438087 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-21712e1b-9cd0-4515-abb8-c3a93344b80f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:45.438676 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-21712e1b-9cd0-4515-abb8-c3a93344b80f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:45.439344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-21712e1b-9cd0-4515-abb8-c3a93344b80f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:45.440102 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-21712e1b-9cd0-4515-abb8-c3a93344b80f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:45.440757 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-21712e1b-9cd0-4515-abb8-c3a93344b80f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:45.441385 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-21712e1b-9cd0-4515-abb8-c3a93344b80f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:45.692565 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-a0cf07ec-976b-4162-b053-e9fddd43b73b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:45.703189 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-a0cf07ec-976b-4162-b053-e9fddd43b73b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.481307 Oct 28 09:45:45.704014 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 66/127] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:45 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:45.713978 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c3b67a1e-3102-4ac1-bf0f-39ffb8c67857 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:45.714809 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c3b67a1e-3102-4ac1-bf0f-39ffb8c67857 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:45.722383 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c3b67a1e-3102-4ac1-bf0f-39ffb8c67857 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:45.723182 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c3b67a1e-3102-4ac1-bf0f-39ffb8c67857 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:45.723957 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c3b67a1e-3102-4ac1-bf0f-39ffb8c67857 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:45.725089 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c3b67a1e-3102-4ac1-bf0f-39ffb8c67857 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:45.725714 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c3b67a1e-3102-4ac1-bf0f-39ffb8c67857 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:45.726255 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c3b67a1e-3102-4ac1-bf0f-39ffb8c67857 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:45.904908 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-21712e1b-9cd0-4515-abb8-c3a93344b80f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:45.917531 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-21712e1b-9cd0-4515-abb8-c3a93344b80f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1314 microversion: 2.1 time: 0.489973 Oct 28 09:45:45.932458 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 62/128] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:45 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1314 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:46.075484 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-acad5e56-2c3f-44ea-bb6e-231428cb224d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:46.075484 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-acad5e56-2c3f-44ea-bb6e-231428cb224d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:46.083148 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-acad5e56-2c3f-44ea-bb6e-231428cb224d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:46.083671 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-acad5e56-2c3f-44ea-bb6e-231428cb224d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:46.084106 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-acad5e56-2c3f-44ea-bb6e-231428cb224d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:46.084801 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-acad5e56-2c3f-44ea-bb6e-231428cb224d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:46.085246 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-acad5e56-2c3f-44ea-bb6e-231428cb224d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:46.085901 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-acad5e56-2c3f-44ea-bb6e-231428cb224d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:46.150000 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c3b67a1e-3102-4ac1-bf0f-39ffb8c67857 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:46.153622 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:46.159196 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c3b67a1e-3102-4ac1-bf0f-39ffb8c67857 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.453169 Oct 28 09:45:46.159871 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 67/129] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:45 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:46.165456 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-902caffc-35f9-465f-a3fc-d89b789a2f1b req-b5c11e42-6f36-4512-8cd5-1855473b39e6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "82790f9d-85c2-4205-b9ff-290df00c2edf", "tag": "392bf558-627c-4e33-82ed-38a82396d7b0"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:46.178578 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-902caffc-35f9-465f-a3fc-d89b789a2f1b req-b5c11e42-6f36-4512-8cd5-1855473b39e6 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:46.179378 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-902caffc-35f9-465f-a3fc-d89b789a2f1b req-b5c11e42-6f36-4512-8cd5-1855473b39e6 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:46.179869 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-902caffc-35f9-465f-a3fc-d89b789a2f1b req-b5c11e42-6f36-4512-8cd5-1855473b39e6 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:46.191987 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-902caffc-35f9-465f-a3fc-d89b789a2f1b req-b5c11e42-6f36-4512-8cd5-1855473b39e6 service nova] Creating event network-changed:392bf558-627c-4e33-82ed-38a82396d7b0 for instance 82790f9d-85c2-4205-b9ff-290df00c2edf on hv-cinder-zed-3 Oct 28 09:45:46.197630 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-902caffc-35f9-465f-a3fc-d89b789a2f1b req-b5c11e42-6f36-4512-8cd5-1855473b39e6 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036314 Oct 28 09:45:46.198358 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 68/130] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:45:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:45:46.273323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d97de69e-ba4c-4878-b574-4c8b28bd1ce1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:46.274269 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d97de69e-ba4c-4878-b574-4c8b28bd1ce1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:46.282682 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d97de69e-ba4c-4878-b574-4c8b28bd1ce1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:46.283400 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d97de69e-ba4c-4878-b574-4c8b28bd1ce1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:46.284134 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d97de69e-ba4c-4878-b574-4c8b28bd1ce1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:46.285432 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d97de69e-ba4c-4878-b574-4c8b28bd1ce1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:46.285943 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d97de69e-ba4c-4878-b574-4c8b28bd1ce1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:46.286463 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d97de69e-ba4c-4878-b574-4c8b28bd1ce1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:46.408669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-acad5e56-2c3f-44ea-bb6e-231428cb224d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:46.418771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-acad5e56-2c3f-44ea-bb6e-231428cb224d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:46.420213 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-acad5e56-2c3f-44ea-bb6e-231428cb224d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:46.421406 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-acad5e56-2c3f-44ea-bb6e-231428cb224d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.352452 Oct 28 09:45:46.422792 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 63/131] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:46 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:45:46.542954 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d97de69e-ba4c-4878-b574-4c8b28bd1ce1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:46.551378 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d97de69e-ba4c-4878-b574-4c8b28bd1ce1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.282983 Oct 28 09:45:46.554434 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 69/132] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:46 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:46.728611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b8bcb68a-c71d-45a7-a98a-6c82290bc4ab tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:46.729927 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b8bcb68a-c71d-45a7-a98a-6c82290bc4ab tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:46.738331 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b8bcb68a-c71d-45a7-a98a-6c82290bc4ab tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:46.739102 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b8bcb68a-c71d-45a7-a98a-6c82290bc4ab tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:46.739728 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b8bcb68a-c71d-45a7-a98a-6c82290bc4ab tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:46.741596 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b8bcb68a-c71d-45a7-a98a-6c82290bc4ab tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:46.742848 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b8bcb68a-c71d-45a7-a98a-6c82290bc4ab tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:46.743459 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b8bcb68a-c71d-45a7-a98a-6c82290bc4ab tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:46.901522 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b8bcb68a-c71d-45a7-a98a-6c82290bc4ab tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:46.909241 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b8bcb68a-c71d-45a7-a98a-6c82290bc4ab tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.185588 Oct 28 09:45:46.909856 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 64/133] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:46 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:46.941490 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-17217327-903c-4b74-8884-63b40976761a tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:46.942091 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-17217327-903c-4b74-8884-63b40976761a tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:46.948189 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-17217327-903c-4b74-8884-63b40976761a tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:46.948663 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-17217327-903c-4b74-8884-63b40976761a tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:46.949088 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-17217327-903c-4b74-8884-63b40976761a tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:46.949762 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-17217327-903c-4b74-8884-63b40976761a tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:46.950202 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-17217327-903c-4b74-8884-63b40976761a tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:46.950651 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-17217327-903c-4b74-8884-63b40976761a tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:47.090006 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-17217327-903c-4b74-8884-63b40976761a tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:47.099063 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-17217327-903c-4b74-8884-63b40976761a tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1314 microversion: 2.1 time: 0.161397 Oct 28 09:45:47.100184 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 70/134] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:46 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:47.187498 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-542fd314-7ebb-4f93-840d-5447851a5dc4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:47.188147 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-542fd314-7ebb-4f93-840d-5447851a5dc4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:47.194648 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-542fd314-7ebb-4f93-840d-5447851a5dc4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:47.195122 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-542fd314-7ebb-4f93-840d-5447851a5dc4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:47.195548 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-542fd314-7ebb-4f93-840d-5447851a5dc4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:47.196253 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-542fd314-7ebb-4f93-840d-5447851a5dc4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:47.196694 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-542fd314-7ebb-4f93-840d-5447851a5dc4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:47.197100 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-542fd314-7ebb-4f93-840d-5447851a5dc4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:47.215622 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=100 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:47.332620 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-542fd314-7ebb-4f93-840d-5447851a5dc4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:47.341318 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-542fd314-7ebb-4f93-840d-5447851a5dc4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.158444 Oct 28 09:45:47.341882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 65/135] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:47 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:47.452599 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c0920a0f-5f6e-485b-8ff1-356c085ea63e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:47.452599 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c0920a0f-5f6e-485b-8ff1-356c085ea63e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:47.459018 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0920a0f-5f6e-485b-8ff1-356c085ea63e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:47.459489 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0920a0f-5f6e-485b-8ff1-356c085ea63e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:47.459924 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0920a0f-5f6e-485b-8ff1-356c085ea63e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:47.460740 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0920a0f-5f6e-485b-8ff1-356c085ea63e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:47.461240 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0920a0f-5f6e-485b-8ff1-356c085ea63e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:47.461784 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0920a0f-5f6e-485b-8ff1-356c085ea63e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:47.575523 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4b3a6cf3-b0fb-470e-8bf7-576eeb35575a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:47.576172 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4b3a6cf3-b0fb-470e-8bf7-576eeb35575a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:47.583129 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b3a6cf3-b0fb-470e-8bf7-576eeb35575a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:47.584355 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b3a6cf3-b0fb-470e-8bf7-576eeb35575a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:47.585156 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b3a6cf3-b0fb-470e-8bf7-576eeb35575a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:47.585966 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b3a6cf3-b0fb-470e-8bf7-576eeb35575a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:47.586452 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b3a6cf3-b0fb-470e-8bf7-576eeb35575a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:47.586883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b3a6cf3-b0fb-470e-8bf7-576eeb35575a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:47.608534 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c0920a0f-5f6e-485b-8ff1-356c085ea63e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:47.616923 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c0920a0f-5f6e-485b-8ff1-356c085ea63e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:47.618344 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c0920a0f-5f6e-485b-8ff1-356c085ea63e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:47.619492 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c0920a0f-5f6e-485b-8ff1-356c085ea63e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.173275 Oct 28 09:45:47.620082 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 71/136] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:47 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:47.717196 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-4b3a6cf3-b0fb-470e-8bf7-576eeb35575a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:47.725317 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4b3a6cf3-b0fb-470e-8bf7-576eeb35575a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.154030 Oct 28 09:45:47.725885 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 66/137] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:47 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:47.936161 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-1ed662bf-0199-4cda-afc0-9fe89e1f23db tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:47.936772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-1ed662bf-0199-4cda-afc0-9fe89e1f23db tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:47.945779 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1ed662bf-0199-4cda-afc0-9fe89e1f23db tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:47.947122 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1ed662bf-0199-4cda-afc0-9fe89e1f23db tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:47.947708 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1ed662bf-0199-4cda-afc0-9fe89e1f23db tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:47.948567 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1ed662bf-0199-4cda-afc0-9fe89e1f23db tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:47.949164 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1ed662bf-0199-4cda-afc0-9fe89e1f23db tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:47.949934 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1ed662bf-0199-4cda-afc0-9fe89e1f23db tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:48.121806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-de18b4c6-44c0-4fe0-8f13-b2fe18125239 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:48.122506 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-de18b4c6-44c0-4fe0-8f13-b2fe18125239 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:48.131735 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-de18b4c6-44c0-4fe0-8f13-b2fe18125239 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:48.143098 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-de18b4c6-44c0-4fe0-8f13-b2fe18125239 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:48.143937 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-de18b4c6-44c0-4fe0-8f13-b2fe18125239 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:48.144867 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-de18b4c6-44c0-4fe0-8f13-b2fe18125239 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:48.145608 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-de18b4c6-44c0-4fe0-8f13-b2fe18125239 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:48.146249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-de18b4c6-44c0-4fe0-8f13-b2fe18125239 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:48.659351 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-1ed662bf-0199-4cda-afc0-9fe89e1f23db tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:48.667994 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-1ed662bf-0199-4cda-afc0-9fe89e1f23db tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.736715 Oct 28 09:45:48.668558 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 72/138] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:47 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 740 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:48.673283 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-7511dc6d-1b80-453b-be2e-6b1e420283b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:48.673894 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-7511dc6d-1b80-453b-be2e-6b1e420283b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:48.680069 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7511dc6d-1b80-453b-be2e-6b1e420283b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:48.680711 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7511dc6d-1b80-453b-be2e-6b1e420283b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:48.681396 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7511dc6d-1b80-453b-be2e-6b1e420283b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:48.682224 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7511dc6d-1b80-453b-be2e-6b1e420283b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:48.682928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7511dc6d-1b80-453b-be2e-6b1e420283b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:48.683674 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7511dc6d-1b80-453b-be2e-6b1e420283b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:48.685965 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-de18b4c6-44c0-4fe0-8f13-b2fe18125239 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:48.694447 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-de18b4c6-44c0-4fe0-8f13-b2fe18125239 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1314 microversion: 2.1 time: 0.576337 Oct 28 09:45:48.697080 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 67/139] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:48 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1314 bytes in 580 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:48.712255 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-b30cc68f-4029-4e53-b874-cddf750918c5 req-d4933d98-f914-4b72-835b-0bb5193cb857 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b981eca6-4d6f-4e48-bf62-016833f2885d", "tag": "e1425374-6966-45a1-bf96-5df1dadc813c"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:48.732482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-b30cc68f-4029-4e53-b874-cddf750918c5 req-d4933d98-f914-4b72-835b-0bb5193cb857 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:48.732974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-b30cc68f-4029-4e53-b874-cddf750918c5 req-d4933d98-f914-4b72-835b-0bb5193cb857 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:48.733403 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-b30cc68f-4029-4e53-b874-cddf750918c5 req-d4933d98-f914-4b72-835b-0bb5193cb857 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:48.748027 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-b30cc68f-4029-4e53-b874-cddf750918c5 req-d4933d98-f914-4b72-835b-0bb5193cb857 service nova] Creating event network-changed:e1425374-6966-45a1-bf96-5df1dadc813c for instance b981eca6-4d6f-4e48-bf62-016833f2885d on hv-cinder-zed-3 Oct 28 09:45:48.754225 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-b30cc68f-4029-4e53-b874-cddf750918c5 req-d4933d98-f914-4b72-835b-0bb5193cb857 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045737 Oct 28 09:45:48.755159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 68/140] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:45:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:45:48.761961 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ae6ef6f9-5963-4259-8d2f-a3b471a6ba8c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:48.763131 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ae6ef6f9-5963-4259-8d2f-a3b471a6ba8c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:48.770561 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae6ef6f9-5963-4259-8d2f-a3b471a6ba8c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:48.772123 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae6ef6f9-5963-4259-8d2f-a3b471a6ba8c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:48.772562 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae6ef6f9-5963-4259-8d2f-a3b471a6ba8c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:48.773337 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae6ef6f9-5963-4259-8d2f-a3b471a6ba8c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:48.773956 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae6ef6f9-5963-4259-8d2f-a3b471a6ba8c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:48.774426 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae6ef6f9-5963-4259-8d2f-a3b471a6ba8c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:48.896625 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7511dc6d-1b80-453b-be2e-6b1e420283b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:48.905296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-7511dc6d-1b80-453b-be2e-6b1e420283b2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.235402 Oct 28 09:45:48.905951 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 73/141] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:48 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:48.915408 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-61bbfcfc-4fa7-4888-93e6-2da640fee748 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:48.916505 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-61bbfcfc-4fa7-4888-93e6-2da640fee748 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:48.923478 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61bbfcfc-4fa7-4888-93e6-2da640fee748 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:48.924280 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61bbfcfc-4fa7-4888-93e6-2da640fee748 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:48.924785 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61bbfcfc-4fa7-4888-93e6-2da640fee748 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:48.925616 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61bbfcfc-4fa7-4888-93e6-2da640fee748 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:48.926163 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61bbfcfc-4fa7-4888-93e6-2da640fee748 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:48.926799 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61bbfcfc-4fa7-4888-93e6-2da640fee748 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:49.023316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ae6ef6f9-5963-4259-8d2f-a3b471a6ba8c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:49.037843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ae6ef6f9-5963-4259-8d2f-a3b471a6ba8c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:49.038330 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ae6ef6f9-5963-4259-8d2f-a3b471a6ba8c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:49.039526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ae6ef6f9-5963-4259-8d2f-a3b471a6ba8c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.282852 Oct 28 09:45:49.040421 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 69/142] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:48 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:49.151760 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-61bbfcfc-4fa7-4888-93e6-2da640fee748 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:49.161145 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-61bbfcfc-4fa7-4888-93e6-2da640fee748 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.253805 Oct 28 09:45:49.163424 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 74/143] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:48 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:49.691024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-556eae25-6a44-4f98-b9d3-67b610e3aa3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:49.691630 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-556eae25-6a44-4f98-b9d3-67b610e3aa3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:49.698276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-556eae25-6a44-4f98-b9d3-67b610e3aa3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:49.698870 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-556eae25-6a44-4f98-b9d3-67b610e3aa3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:49.699451 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-556eae25-6a44-4f98-b9d3-67b610e3aa3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:49.700199 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-556eae25-6a44-4f98-b9d3-67b610e3aa3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:49.700646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-556eae25-6a44-4f98-b9d3-67b610e3aa3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:49.701063 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-556eae25-6a44-4f98-b9d3-67b610e3aa3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:49.722161 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-cc5c5ddd-a3e6-4490-898c-b294e232713c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:49.723101 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-cc5c5ddd-a3e6-4490-898c-b294e232713c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:49.730420 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cc5c5ddd-a3e6-4490-898c-b294e232713c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:49.731111 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cc5c5ddd-a3e6-4490-898c-b294e232713c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:49.731968 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cc5c5ddd-a3e6-4490-898c-b294e232713c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:49.732998 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cc5c5ddd-a3e6-4490-898c-b294e232713c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:49.733580 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cc5c5ddd-a3e6-4490-898c-b294e232713c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:49.734162 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cc5c5ddd-a3e6-4490-898c-b294e232713c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:49.941259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-556eae25-6a44-4f98-b9d3-67b610e3aa3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:49.949142 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-cc5c5ddd-a3e6-4490-898c-b294e232713c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:49.959564 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-cc5c5ddd-a3e6-4490-898c-b294e232713c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1314 microversion: 2.1 time: 0.244579 Oct 28 09:45:49.960133 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 75/144] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:49 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1314 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:49.964664 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-09413458-0765-45cc-b8eb-22b2c1767e2c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:49.965415 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-09413458-0765-45cc-b8eb-22b2c1767e2c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:49.971465 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-09413458-0765-45cc-b8eb-22b2c1767e2c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:49.972121 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-09413458-0765-45cc-b8eb-22b2c1767e2c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:49.972731 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-09413458-0765-45cc-b8eb-22b2c1767e2c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:49.973785 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-09413458-0765-45cc-b8eb-22b2c1767e2c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:49.974975 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-09413458-0765-45cc-b8eb-22b2c1767e2c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:49.975797 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-09413458-0765-45cc-b8eb-22b2c1767e2c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:49.976910 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-556eae25-6a44-4f98-b9d3-67b610e3aa3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.290165 Oct 28 09:45:49.977777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 70/145] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:49 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:50.064167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-20b0f3aa-f686-43d6-ac1a-83cf0e23acbe tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:50.064769 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-20b0f3aa-f686-43d6-ac1a-83cf0e23acbe tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:50.067979 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=98 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:50.072339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-20b0f3aa-f686-43d6-ac1a-83cf0e23acbe tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:50.073165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-20b0f3aa-f686-43d6-ac1a-83cf0e23acbe tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:50.073250 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-20b0f3aa-f686-43d6-ac1a-83cf0e23acbe tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:50.073917 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-20b0f3aa-f686-43d6-ac1a-83cf0e23acbe tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:50.074360 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-20b0f3aa-f686-43d6-ac1a-83cf0e23acbe tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:50.074806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-20b0f3aa-f686-43d6-ac1a-83cf0e23acbe tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:50.135808 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-09413458-0765-45cc-b8eb-22b2c1767e2c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:50.145097 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-09413458-0765-45cc-b8eb-22b2c1767e2c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.183824 Oct 28 09:45:50.145881 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 76/146] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:49 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:50.190158 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-96012064-ed9d-4c34-b0be-35f61c2c7b64 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:50.190800 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-96012064-ed9d-4c34-b0be-35f61c2c7b64 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:50.197685 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-96012064-ed9d-4c34-b0be-35f61c2c7b64 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:50.198444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-96012064-ed9d-4c34-b0be-35f61c2c7b64 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:50.198935 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-96012064-ed9d-4c34-b0be-35f61c2c7b64 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:50.199816 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-96012064-ed9d-4c34-b0be-35f61c2c7b64 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:50.200421 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-96012064-ed9d-4c34-b0be-35f61c2c7b64 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:50.200837 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-96012064-ed9d-4c34-b0be-35f61c2c7b64 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:50.246191 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-20b0f3aa-f686-43d6-ac1a-83cf0e23acbe tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:50.268903 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-20b0f3aa-f686-43d6-ac1a-83cf0e23acbe tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:50.270448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-20b0f3aa-f686-43d6-ac1a-83cf0e23acbe tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:50.272122 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-20b0f3aa-f686-43d6-ac1a-83cf0e23acbe tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.211823 Oct 28 09:45:50.272798 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 71/147] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:50 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:45:50.370531 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-96012064-ed9d-4c34-b0be-35f61c2c7b64 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:50.378909 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-96012064-ed9d-4c34-b0be-35f61c2c7b64 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.193231 Oct 28 09:45:50.379480 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 77/148] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:50 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:50.985124 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-568249b7-6eef-4485-a0c0-ab86122a626f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:50.986596 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-568249b7-6eef-4485-a0c0-ab86122a626f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:50.993158 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-568249b7-6eef-4485-a0c0-ab86122a626f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:50.995235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-568249b7-6eef-4485-a0c0-ab86122a626f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:50.995235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-568249b7-6eef-4485-a0c0-ab86122a626f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:50.995235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-568249b7-6eef-4485-a0c0-ab86122a626f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:50.995528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-568249b7-6eef-4485-a0c0-ab86122a626f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:50.995927 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-568249b7-6eef-4485-a0c0-ab86122a626f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:51.001838 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-7a940a3e-faae-4d99-b5f5-8d83b329ff6d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:51.002538 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-7a940a3e-faae-4d99-b5f5-8d83b329ff6d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:51.010412 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7a940a3e-faae-4d99-b5f5-8d83b329ff6d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:51.010903 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7a940a3e-faae-4d99-b5f5-8d83b329ff6d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:51.011526 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7a940a3e-faae-4d99-b5f5-8d83b329ff6d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:51.012215 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7a940a3e-faae-4d99-b5f5-8d83b329ff6d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:51.012658 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7a940a3e-faae-4d99-b5f5-8d83b329ff6d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:51.013279 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7a940a3e-faae-4d99-b5f5-8d83b329ff6d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:51.208630 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-568249b7-6eef-4485-a0c0-ab86122a626f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:51.215880 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7a940a3e-faae-4d99-b5f5-8d83b329ff6d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:51.217944 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-568249b7-6eef-4485-a0c0-ab86122a626f tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1314 microversion: 2.1 time: 0.236311 Oct 28 09:45:51.218558 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 72/149] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:50 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1314 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:51.225822 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-241c808f-1e77-4753-ba4a-3f3b8390e70d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:51.226176 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-7a940a3e-faae-4d99-b5f5-8d83b329ff6d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.228982 Oct 28 09:45:51.226743 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 78/150] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:50 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:51.237076 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-241c808f-1e77-4753-ba4a-3f3b8390e70d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:51.245355 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-241c808f-1e77-4753-ba4a-3f3b8390e70d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:51.245846 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-241c808f-1e77-4753-ba4a-3f3b8390e70d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:51.246296 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-241c808f-1e77-4753-ba4a-3f3b8390e70d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:51.247051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-241c808f-1e77-4753-ba4a-3f3b8390e70d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:51.247505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-241c808f-1e77-4753-ba4a-3f3b8390e70d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:51.247924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-241c808f-1e77-4753-ba4a-3f3b8390e70d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:51.297594 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-05459b09-3a84-416e-821a-218f595b957c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:51.300420 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-05459b09-3a84-416e-821a-218f595b957c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:51.311452 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:51.330613 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-05459b09-3a84-416e-821a-218f595b957c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:51.331262 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-05459b09-3a84-416e-821a-218f595b957c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:51.331712 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-05459b09-3a84-416e-821a-218f595b957c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:51.332537 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-05459b09-3a84-416e-821a-218f595b957c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:51.333099 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-05459b09-3a84-416e-821a-218f595b957c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:51.333593 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-05459b09-3a84-416e-821a-218f595b957c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:51.668819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-241c808f-1e77-4753-ba4a-3f3b8390e70d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:51.682975 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-241c808f-1e77-4753-ba4a-3f3b8390e70d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.463158 Oct 28 09:45:51.690469 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 73/151] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:51 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:51.690469 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-dd400d10-1f5e-488c-9b46-13e4815200e0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:51.690469 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-dd400d10-1f5e-488c-9b46-13e4815200e0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:51.706310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dd400d10-1f5e-488c-9b46-13e4815200e0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:51.706841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dd400d10-1f5e-488c-9b46-13e4815200e0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:51.707272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dd400d10-1f5e-488c-9b46-13e4815200e0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:51.707969 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dd400d10-1f5e-488c-9b46-13e4815200e0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:51.708439 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dd400d10-1f5e-488c-9b46-13e4815200e0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:51.709351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dd400d10-1f5e-488c-9b46-13e4815200e0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:51.912112 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-05459b09-3a84-416e-821a-218f595b957c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:51.921558 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-05459b09-3a84-416e-821a-218f595b957c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:51.923497 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-05459b09-3a84-416e-821a-218f595b957c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:51.924786 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-05459b09-3a84-416e-821a-218f595b957c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.631249 Oct 28 09:45:51.925423 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 79/152] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:51 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 632 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:45:51.945552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-2748c9c7-5ec5-44dd-bf4f-97f80f12de80 req-6f4b03ee-c56d-438c-878a-0f73ebe9d00e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7", "name": "network-vif-plugged", "status": "completed", "tag": "a69a90a6-23e5-4976-90d8-d36b10ff6b35"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:51.955510 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-2748c9c7-5ec5-44dd-bf4f-97f80f12de80 req-6f4b03ee-c56d-438c-878a-0f73ebe9d00e service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:51.956090 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-2748c9c7-5ec5-44dd-bf4f-97f80f12de80 req-6f4b03ee-c56d-438c-878a-0f73ebe9d00e service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:51.956643 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-2748c9c7-5ec5-44dd-bf4f-97f80f12de80 req-6f4b03ee-c56d-438c-878a-0f73ebe9d00e service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:51.968647 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-2748c9c7-5ec5-44dd-bf4f-97f80f12de80 req-6f4b03ee-c56d-438c-878a-0f73ebe9d00e service nova] Creating event network-vif-plugged:a69a90a6-23e5-4976-90d8-d36b10ff6b35 for instance cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 on hv-cinder-zed-3 Oct 28 09:45:51.973724 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-2748c9c7-5ec5-44dd-bf4f-97f80f12de80 req-6f4b03ee-c56d-438c-878a-0f73ebe9d00e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041760 Oct 28 09:45:51.974485 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 80/153] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:45:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:45:52.128640 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-dd400d10-1f5e-488c-9b46-13e4815200e0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:52.137966 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-dd400d10-1f5e-488c-9b46-13e4815200e0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.453023 Oct 28 09:45:52.138565 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 74/154] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:51 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:52.242012 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5e525b5f-1f44-4120-a83e-336676766529 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:52.242729 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5e525b5f-1f44-4120-a83e-336676766529 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:52.249127 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e525b5f-1f44-4120-a83e-336676766529 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:52.249620 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e525b5f-1f44-4120-a83e-336676766529 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:52.250051 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e525b5f-1f44-4120-a83e-336676766529 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:52.250365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-66721a08-e8cc-4b0a-985f-e977bb0669fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:52.250809 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e525b5f-1f44-4120-a83e-336676766529 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:52.250997 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-66721a08-e8cc-4b0a-985f-e977bb0669fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:52.251255 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e525b5f-1f44-4120-a83e-336676766529 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:52.251832 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e525b5f-1f44-4120-a83e-336676766529 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:52.257976 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-66721a08-e8cc-4b0a-985f-e977bb0669fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:52.258483 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-66721a08-e8cc-4b0a-985f-e977bb0669fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:52.259083 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-66721a08-e8cc-4b0a-985f-e977bb0669fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:52.260022 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-66721a08-e8cc-4b0a-985f-e977bb0669fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:52.260464 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-66721a08-e8cc-4b0a-985f-e977bb0669fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:52.260891 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-66721a08-e8cc-4b0a-985f-e977bb0669fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:52.460455 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5e525b5f-1f44-4120-a83e-336676766529 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:52.466577 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-66721a08-e8cc-4b0a-985f-e977bb0669fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:52.471033 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5e525b5f-1f44-4120-a83e-336676766529 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1314 microversion: 2.1 time: 0.233150 Oct 28 09:45:52.477806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-66721a08-e8cc-4b0a-985f-e977bb0669fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.231252 Oct 28 09:45:52.478358 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 75/155] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:52 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:52.483628 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 81/156] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:52 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1314 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:52.709305 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a856e74b-cbfc-409a-8753-e645ed902cdf tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:52.710076 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a856e74b-cbfc-409a-8753-e645ed902cdf tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:52.716386 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a856e74b-cbfc-409a-8753-e645ed902cdf tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:52.717094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a856e74b-cbfc-409a-8753-e645ed902cdf tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:52.717781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a856e74b-cbfc-409a-8753-e645ed902cdf tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:52.718574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a856e74b-cbfc-409a-8753-e645ed902cdf tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:52.719133 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a856e74b-cbfc-409a-8753-e645ed902cdf tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:52.719756 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a856e74b-cbfc-409a-8753-e645ed902cdf tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:52.869499 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a856e74b-cbfc-409a-8753-e645ed902cdf tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:52.881620 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a856e74b-cbfc-409a-8753-e645ed902cdf tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.177042 Oct 28 09:45:52.881945 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 76/157] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:52 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:52.951699 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-a954c773-fdc4-4f54-ba0f-d751001546af tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:52.952522 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-a954c773-fdc4-4f54-ba0f-d751001546af tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:52.960201 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a954c773-fdc4-4f54-ba0f-d751001546af tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:52.960808 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a954c773-fdc4-4f54-ba0f-d751001546af tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:52.961475 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a954c773-fdc4-4f54-ba0f-d751001546af tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:52.962656 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a954c773-fdc4-4f54-ba0f-d751001546af tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:52.963461 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a954c773-fdc4-4f54-ba0f-d751001546af tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:52.964536 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a954c773-fdc4-4f54-ba0f-d751001546af tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:53.163378 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5eef39eb-1377-4367-95df-7da9fafb6a65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:53.164647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5eef39eb-1377-4367-95df-7da9fafb6a65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:53.172944 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5eef39eb-1377-4367-95df-7da9fafb6a65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:53.173603 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5eef39eb-1377-4367-95df-7da9fafb6a65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:53.174111 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5eef39eb-1377-4367-95df-7da9fafb6a65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:53.174951 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5eef39eb-1377-4367-95df-7da9fafb6a65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:53.175491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5eef39eb-1377-4367-95df-7da9fafb6a65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:53.176003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5eef39eb-1377-4367-95df-7da9fafb6a65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:53.243046 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-a954c773-fdc4-4f54-ba0f-d751001546af tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:53.256262 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-a954c773-fdc4-4f54-ba0f-d751001546af tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:53.259062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-a954c773-fdc4-4f54-ba0f-d751001546af tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:53.261100 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-a954c773-fdc4-4f54-ba0f-d751001546af tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.314434 Oct 28 09:45:53.262461 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 82/158] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:52 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:53.426930 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5eef39eb-1377-4367-95df-7da9fafb6a65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:53.435740 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5eef39eb-1377-4367-95df-7da9fafb6a65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.276825 Oct 28 09:45:53.436484 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 77/159] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:53 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:53.495220 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-20bfbca7-65bf-4af8-a026-dcd253ab5aaa tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:53.496505 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-20bfbca7-65bf-4af8-a026-dcd253ab5aaa tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:53.501337 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d868a2ac-a8bf-497d-ab6d-8e0dfd63a445 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:53.501943 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d868a2ac-a8bf-497d-ab6d-8e0dfd63a445 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:53.503561 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-20bfbca7-65bf-4af8-a026-dcd253ab5aaa tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:53.504138 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-20bfbca7-65bf-4af8-a026-dcd253ab5aaa tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:53.504798 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-20bfbca7-65bf-4af8-a026-dcd253ab5aaa tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:53.505868 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-20bfbca7-65bf-4af8-a026-dcd253ab5aaa tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:53.506733 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-20bfbca7-65bf-4af8-a026-dcd253ab5aaa tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:53.507223 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-20bfbca7-65bf-4af8-a026-dcd253ab5aaa tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:53.509506 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d868a2ac-a8bf-497d-ab6d-8e0dfd63a445 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:53.509992 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d868a2ac-a8bf-497d-ab6d-8e0dfd63a445 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:53.510432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d868a2ac-a8bf-497d-ab6d-8e0dfd63a445 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:53.519816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d868a2ac-a8bf-497d-ab6d-8e0dfd63a445 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:53.523276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d868a2ac-a8bf-497d-ab6d-8e0dfd63a445 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:53.524994 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d868a2ac-a8bf-497d-ab6d-8e0dfd63a445 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:53.933793 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d868a2ac-a8bf-497d-ab6d-8e0dfd63a445 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:53.945871 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d868a2ac-a8bf-497d-ab6d-8e0dfd63a445 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.448772 Oct 28 09:45:53.948353 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 78/160] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:53 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:53.953164 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-aa83f851-aecd-4235-9fd5-59c48d2da22a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:53.953770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-aa83f851-aecd-4235-9fd5-59c48d2da22a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:53.961436 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-aa83f851-aecd-4235-9fd5-59c48d2da22a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:53.967208 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-20bfbca7-65bf-4af8-a026-dcd253ab5aaa tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:53.976337 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-20bfbca7-65bf-4af8-a026-dcd253ab5aaa tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1314 microversion: 2.1 time: 0.485038 Oct 28 09:45:53.976939 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-aa83f851-aecd-4235-9fd5-59c48d2da22a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:53.977848 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-aa83f851-aecd-4235-9fd5-59c48d2da22a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:53.978629 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-aa83f851-aecd-4235-9fd5-59c48d2da22a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:53.979094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-aa83f851-aecd-4235-9fd5-59c48d2da22a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:53.979517 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-aa83f851-aecd-4235-9fd5-59c48d2da22a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:53.981458 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 83/161] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:53 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1314 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:54.005131 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-42efb8da-08a0-4769-a73c-1aa6427db523 req-5c5b4d96-16b1-46c2-8057-34a9cee3c9aa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "088d0e80-3838-4ada-938b-f26f194d5abc", "name": "network-vif-plugged", "status": "completed", "tag": "a903c483-3505-4c5d-9652-f34609414d9d"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:54.031508 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-42efb8da-08a0-4769-a73c-1aa6427db523 req-5c5b4d96-16b1-46c2-8057-34a9cee3c9aa service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:54.033525 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-42efb8da-08a0-4769-a73c-1aa6427db523 req-5c5b4d96-16b1-46c2-8057-34a9cee3c9aa service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:54.034353 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-42efb8da-08a0-4769-a73c-1aa6427db523 req-5c5b4d96-16b1-46c2-8057-34a9cee3c9aa service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:54.051430 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-42efb8da-08a0-4769-a73c-1aa6427db523 req-5c5b4d96-16b1-46c2-8057-34a9cee3c9aa service nova] Creating event network-vif-plugged:a903c483-3505-4c5d-9652-f34609414d9d for instance 088d0e80-3838-4ada-938b-f26f194d5abc on hv-cinder-zed-3 Oct 28 09:45:54.057829 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-42efb8da-08a0-4769-a73c-1aa6427db523 req-5c5b4d96-16b1-46c2-8057-34a9cee3c9aa service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055846 Oct 28 09:45:54.059863 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 84/162] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:45:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:45:54.152689 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-aa83f851-aecd-4235-9fd5-59c48d2da22a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:54.162837 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-aa83f851-aecd-4235-9fd5-59c48d2da22a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.213214 Oct 28 09:45:54.164261 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 79/163] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:53 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:54.288494 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2acc31e0-c923-4b64-87f7-cb7b955fbb78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:54.289315 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2acc31e0-c923-4b64-87f7-cb7b955fbb78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:54.297458 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2acc31e0-c923-4b64-87f7-cb7b955fbb78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:54.298196 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2acc31e0-c923-4b64-87f7-cb7b955fbb78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:54.298884 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2acc31e0-c923-4b64-87f7-cb7b955fbb78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:54.299877 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2acc31e0-c923-4b64-87f7-cb7b955fbb78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:54.300780 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2acc31e0-c923-4b64-87f7-cb7b955fbb78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:54.301497 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2acc31e0-c923-4b64-87f7-cb7b955fbb78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:54.459818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-562aab69-5dec-4186-a8a1-482a7b6bf4b5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:54.461013 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-562aab69-5dec-4186-a8a1-482a7b6bf4b5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:54.468155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-562aab69-5dec-4186-a8a1-482a7b6bf4b5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:54.469041 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-562aab69-5dec-4186-a8a1-482a7b6bf4b5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:54.469697 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-562aab69-5dec-4186-a8a1-482a7b6bf4b5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:54.470632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-562aab69-5dec-4186-a8a1-482a7b6bf4b5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:54.471288 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-562aab69-5dec-4186-a8a1-482a7b6bf4b5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:54.471904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-562aab69-5dec-4186-a8a1-482a7b6bf4b5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:54.752962 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2acc31e0-c923-4b64-87f7-cb7b955fbb78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:54.781466 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2acc31e0-c923-4b64-87f7-cb7b955fbb78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:54.783855 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2acc31e0-c923-4b64-87f7-cb7b955fbb78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:54.787879 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2acc31e0-c923-4b64-87f7-cb7b955fbb78 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.505355 Oct 28 09:45:54.806365 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 85/164] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:54 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:54.879660 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-562aab69-5dec-4186-a8a1-482a7b6bf4b5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:54.894616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-562aab69-5dec-4186-a8a1-482a7b6bf4b5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.439327 Oct 28 09:45:54.910858 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 80/165] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:54 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:54.975747 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-7d054643-4d6e-4699-8d5f-6cb4d5f667f5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:54.989184 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-7d054643-4d6e-4699-8d5f-6cb4d5f667f5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:55.004374 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7d054643-4d6e-4699-8d5f-6cb4d5f667f5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:55.004868 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7d054643-4d6e-4699-8d5f-6cb4d5f667f5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:55.007909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-cd99b19a-168e-4242-bdff-21d8aba9a4f2 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:55.008729 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-cd99b19a-168e-4242-bdff-21d8aba9a4f2 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:55.011160 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7d054643-4d6e-4699-8d5f-6cb4d5f667f5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:55.011960 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7d054643-4d6e-4699-8d5f-6cb4d5f667f5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:55.013366 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7d054643-4d6e-4699-8d5f-6cb4d5f667f5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:55.013952 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7d054643-4d6e-4699-8d5f-6cb4d5f667f5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:55.017534 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd99b19a-168e-4242-bdff-21d8aba9a4f2 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:55.018971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd99b19a-168e-4242-bdff-21d8aba9a4f2 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:55.019456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd99b19a-168e-4242-bdff-21d8aba9a4f2 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:55.020284 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd99b19a-168e-4242-bdff-21d8aba9a4f2 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:55.021805 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd99b19a-168e-4242-bdff-21d8aba9a4f2 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:55.022230 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd99b19a-168e-4242-bdff-21d8aba9a4f2 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:55.062518 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:55.410088 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7d054643-4d6e-4699-8d5f-6cb4d5f667f5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:55.419899 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-7d054643-4d6e-4699-8d5f-6cb4d5f667f5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.452740 Oct 28 09:45:55.422892 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 86/166] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:54 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:55.440117 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-df5eaf63-29a0-4a8e-8c06-2ce37ec6792d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:55.445290 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-df5eaf63-29a0-4a8e-8c06-2ce37ec6792d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:55.464136 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-df5eaf63-29a0-4a8e-8c06-2ce37ec6792d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:55.464634 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-df5eaf63-29a0-4a8e-8c06-2ce37ec6792d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:55.465060 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-df5eaf63-29a0-4a8e-8c06-2ce37ec6792d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:55.465751 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-df5eaf63-29a0-4a8e-8c06-2ce37ec6792d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:55.466186 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-df5eaf63-29a0-4a8e-8c06-2ce37ec6792d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:55.466634 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-df5eaf63-29a0-4a8e-8c06-2ce37ec6792d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:55.481431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-cd99b19a-168e-4242-bdff-21d8aba9a4f2 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:55.495195 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-cd99b19a-168e-4242-bdff-21d8aba9a4f2 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1314 microversion: 2.1 time: 0.491127 Oct 28 09:45:55.496067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 81/167] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:55 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1314 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:55.722228 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-df5eaf63-29a0-4a8e-8c06-2ce37ec6792d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:55.735107 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-df5eaf63-29a0-4a8e-8c06-2ce37ec6792d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.308844 Oct 28 09:45:55.735938 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 87/168] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:55 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:55.831832 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0cdbdc3e-289a-45a6-ad5f-2baadcf152b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:55.832924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0cdbdc3e-289a-45a6-ad5f-2baadcf152b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:55.843661 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0cdbdc3e-289a-45a6-ad5f-2baadcf152b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:55.846821 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0cdbdc3e-289a-45a6-ad5f-2baadcf152b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:55.847510 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0cdbdc3e-289a-45a6-ad5f-2baadcf152b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:55.848281 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0cdbdc3e-289a-45a6-ad5f-2baadcf152b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:55.848739 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0cdbdc3e-289a-45a6-ad5f-2baadcf152b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:55.849622 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0cdbdc3e-289a-45a6-ad5f-2baadcf152b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:55.917822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-651662a9-c782-409d-ad86-342787da914f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:55.918457 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-651662a9-c782-409d-ad86-342787da914f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:55.932748 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-651662a9-c782-409d-ad86-342787da914f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:55.932748 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-651662a9-c782-409d-ad86-342787da914f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:55.932748 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-651662a9-c782-409d-ad86-342787da914f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:55.933263 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-651662a9-c782-409d-ad86-342787da914f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:55.938852 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-651662a9-c782-409d-ad86-342787da914f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:55.939597 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-651662a9-c782-409d-ad86-342787da914f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:56.133866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0cdbdc3e-289a-45a6-ad5f-2baadcf152b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:56.137752 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=100 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:56.143540 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0cdbdc3e-289a-45a6-ad5f-2baadcf152b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:56.145265 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0cdbdc3e-289a-45a6-ad5f-2baadcf152b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:56.147148 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0cdbdc3e-289a-45a6-ad5f-2baadcf152b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.319359 Oct 28 09:45:56.150511 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 82/169] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:55 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:56.153138 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-f103750a-5dab-4b4c-b8f7-718ccdceff0b req-2711f1d7-56bb-440b-837d-b50af9432c2a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c716d5fd-f257-4f6a-8f52-18ad59e5c31e", "name": "network-vif-plugged", "status": "completed", "tag": "c2310b44-82bb-408f-85b5-8a5556b92e25"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:56.163563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-f103750a-5dab-4b4c-b8f7-718ccdceff0b req-2711f1d7-56bb-440b-837d-b50af9432c2a service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:56.164259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-f103750a-5dab-4b4c-b8f7-718ccdceff0b req-2711f1d7-56bb-440b-837d-b50af9432c2a service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:56.164822 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-f103750a-5dab-4b4c-b8f7-718ccdceff0b req-2711f1d7-56bb-440b-837d-b50af9432c2a service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:56.176709 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-f103750a-5dab-4b4c-b8f7-718ccdceff0b req-2711f1d7-56bb-440b-837d-b50af9432c2a service nova] Creating event network-vif-plugged:c2310b44-82bb-408f-85b5-8a5556b92e25 for instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e on hv-cinder-zed-3 Oct 28 09:45:56.182177 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-f103750a-5dab-4b4c-b8f7-718ccdceff0b req-2711f1d7-56bb-440b-837d-b50af9432c2a service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032851 Oct 28 09:45:56.182819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 83/170] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:45:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:45:56.188845 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-651662a9-c782-409d-ad86-342787da914f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:56.197309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-651662a9-c782-409d-ad86-342787da914f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.283901 Oct 28 09:45:56.197875 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 88/171] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:55 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:56.449913 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a6ee3738-e9ac-4f58-a1bf-8ac6003ce1ee tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:56.450629 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a6ee3738-e9ac-4f58-a1bf-8ac6003ce1ee tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:56.458165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6ee3738-e9ac-4f58-a1bf-8ac6003ce1ee tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:56.459424 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6ee3738-e9ac-4f58-a1bf-8ac6003ce1ee tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:56.459851 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6ee3738-e9ac-4f58-a1bf-8ac6003ce1ee tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:56.460539 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6ee3738-e9ac-4f58-a1bf-8ac6003ce1ee tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:56.460978 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6ee3738-e9ac-4f58-a1bf-8ac6003ce1ee tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:56.461388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6ee3738-e9ac-4f58-a1bf-8ac6003ce1ee tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:56.521617 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-ab31b03a-1405-45c8-8be6-3acc554c31ce tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:56.522258 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-ab31b03a-1405-45c8-8be6-3acc554c31ce tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:56.528850 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ab31b03a-1405-45c8-8be6-3acc554c31ce tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:56.529401 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ab31b03a-1405-45c8-8be6-3acc554c31ce tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:56.530276 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ab31b03a-1405-45c8-8be6-3acc554c31ce tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:56.530673 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ab31b03a-1405-45c8-8be6-3acc554c31ce tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:56.531189 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ab31b03a-1405-45c8-8be6-3acc554c31ce tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:56.531670 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ab31b03a-1405-45c8-8be6-3acc554c31ce tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:56.659746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a6ee3738-e9ac-4f58-a1bf-8ac6003ce1ee tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:56.695348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a6ee3738-e9ac-4f58-a1bf-8ac6003ce1ee tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1322 microversion: 2.1 time: 0.249750 Oct 28 09:45:56.696880 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 84/172] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:56 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1322 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:56.698074 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-ab31b03a-1405-45c8-8be6-3acc554c31ce tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:56.706844 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-ab31b03a-1405-45c8-8be6-3acc554c31ce tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1314 microversion: 2.1 time: 0.189568 Oct 28 09:45:56.707931 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 89/173] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:56 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1314 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:56.762549 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-bc70dcfd-c86c-41a0-a7d3-2f5829ad9f7c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:56.763396 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-bc70dcfd-c86c-41a0-a7d3-2f5829ad9f7c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:56.769786 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bc70dcfd-c86c-41a0-a7d3-2f5829ad9f7c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:56.770263 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bc70dcfd-c86c-41a0-a7d3-2f5829ad9f7c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:56.770718 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bc70dcfd-c86c-41a0-a7d3-2f5829ad9f7c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:56.771470 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bc70dcfd-c86c-41a0-a7d3-2f5829ad9f7c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:56.771914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bc70dcfd-c86c-41a0-a7d3-2f5829ad9f7c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:56.772331 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bc70dcfd-c86c-41a0-a7d3-2f5829ad9f7c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:56.919405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-bc70dcfd-c86c-41a0-a7d3-2f5829ad9f7c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:56.929826 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-bc70dcfd-c86c-41a0-a7d3-2f5829ad9f7c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.170194 Oct 28 09:45:56.930477 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 85/174] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:56 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:57.173342 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-997755a1-0d4d-4f19-8551-fe7b8b20d47a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:57.174096 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-997755a1-0d4d-4f19-8551-fe7b8b20d47a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:57.181330 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-997755a1-0d4d-4f19-8551-fe7b8b20d47a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:57.182202 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-997755a1-0d4d-4f19-8551-fe7b8b20d47a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:57.182784 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-997755a1-0d4d-4f19-8551-fe7b8b20d47a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:57.184020 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-997755a1-0d4d-4f19-8551-fe7b8b20d47a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:57.185226 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-997755a1-0d4d-4f19-8551-fe7b8b20d47a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:57.185681 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-997755a1-0d4d-4f19-8551-fe7b8b20d47a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:57.220910 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d1eb9f0a-2e5a-45ae-b852-465a681f173a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:57.221620 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d1eb9f0a-2e5a-45ae-b852-465a681f173a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:57.230575 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d1eb9f0a-2e5a-45ae-b852-465a681f173a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:57.231068 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d1eb9f0a-2e5a-45ae-b852-465a681f173a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:57.231491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d1eb9f0a-2e5a-45ae-b852-465a681f173a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:57.232190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d1eb9f0a-2e5a-45ae-b852-465a681f173a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:57.238653 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d1eb9f0a-2e5a-45ae-b852-465a681f173a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:57.239260 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d1eb9f0a-2e5a-45ae-b852-465a681f173a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:57.429354 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-997755a1-0d4d-4f19-8551-fe7b8b20d47a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:57.439416 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-997755a1-0d4d-4f19-8551-fe7b8b20d47a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:57.440859 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-997755a1-0d4d-4f19-8551-fe7b8b20d47a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:57.442241 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-997755a1-0d4d-4f19-8551-fe7b8b20d47a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.273104 Oct 28 09:45:57.443040 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 90/175] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:57 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:57.450650 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d1eb9f0a-2e5a-45ae-b852-465a681f173a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:57.466614 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d1eb9f0a-2e5a-45ae-b852-465a681f173a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.249460 Oct 28 09:45:57.467141 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 86/176] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:57 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:57.719596 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d516edce-a569-471d-bd52-fd80b2f6aaf9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:57.720122 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d516edce-a569-471d-bd52-fd80b2f6aaf9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:57.726836 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d516edce-a569-471d-bd52-fd80b2f6aaf9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:57.727317 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d516edce-a569-471d-bd52-fd80b2f6aaf9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:57.727751 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d516edce-a569-471d-bd52-fd80b2f6aaf9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:57.728434 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d516edce-a569-471d-bd52-fd80b2f6aaf9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:57.728881 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d516edce-a569-471d-bd52-fd80b2f6aaf9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:57.729296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d516edce-a569-471d-bd52-fd80b2f6aaf9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:57.730678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-34e2daf8-df15-4484-bcee-f840313a8027 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:57.731286 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-34e2daf8-df15-4484-bcee-f840313a8027 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:57.737141 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-34e2daf8-df15-4484-bcee-f840313a8027 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:57.737632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-34e2daf8-df15-4484-bcee-f840313a8027 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:57.738036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-34e2daf8-df15-4484-bcee-f840313a8027 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:57.738776 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-34e2daf8-df15-4484-bcee-f840313a8027 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:57.739222 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-34e2daf8-df15-4484-bcee-f840313a8027 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:57.739639 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-34e2daf8-df15-4484-bcee-f840313a8027 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:57.925058 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d516edce-a569-471d-bd52-fd80b2f6aaf9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:57.928861 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-34e2daf8-df15-4484-bcee-f840313a8027 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:57.936033 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d516edce-a569-471d-bd52-fd80b2f6aaf9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1516 microversion: 2.1 time: 0.220054 Oct 28 09:45:57.936609 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 91/177] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:57 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1516 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:57.937769 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-34e2daf8-df15-4484-bcee-f840313a8027 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1501 microversion: 2.1 time: 0.210824 Oct 28 09:45:57.938343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 87/178] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:57 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1501 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:45:57.952889 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e277b1ec-ad59-4d71-b2ac-851c07d2da20 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:57.953580 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e277b1ec-ad59-4d71-b2ac-851c07d2da20 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:57.962684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ab059505-a394-427b-b85c-6f99b5da2f95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:57.964925 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ab059505-a394-427b-b85c-6f99b5da2f95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:57.969870 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e277b1ec-ad59-4d71-b2ac-851c07d2da20 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:57.970294 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e277b1ec-ad59-4d71-b2ac-851c07d2da20 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:57.970757 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e277b1ec-ad59-4d71-b2ac-851c07d2da20 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:57.971454 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e277b1ec-ad59-4d71-b2ac-851c07d2da20 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:57.971900 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e277b1ec-ad59-4d71-b2ac-851c07d2da20 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:57.972320 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e277b1ec-ad59-4d71-b2ac-851c07d2da20 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:57.981277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab059505-a394-427b-b85c-6f99b5da2f95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:57.981937 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab059505-a394-427b-b85c-6f99b5da2f95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:57.982551 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab059505-a394-427b-b85c-6f99b5da2f95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:57.983432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab059505-a394-427b-b85c-6f99b5da2f95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:57.984041 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab059505-a394-427b-b85c-6f99b5da2f95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:57.984609 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab059505-a394-427b-b85c-6f99b5da2f95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:58.063994 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-ab059505-a394-427b-b85c-6f99b5da2f95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lazy-loading 'flavor' on Instance uuid cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:45:58.167672 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e277b1ec-ad59-4d71-b2ac-851c07d2da20 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:58.176877 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e277b1ec-ad59-4d71-b2ac-851c07d2da20 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.228182 Oct 28 09:45:58.193728 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 92/179] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:57 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:58.205764 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-34b3f5ec-471c-44c8-8e98-adae3e286822 req-c1737671-190d-4c0d-bb0a-29a360407458 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b981eca6-4d6f-4e48-bf62-016833f2885d", "name": "network-vif-plugged", "status": "completed", "tag": "e1425374-6966-45a1-bf96-5df1dadc813c"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:45:58.212286 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ab059505-a394-427b-b85c-6f99b5da2f95 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "POST /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7/action" status: 200 len: 32 microversion: 2.1 time: 0.243348 Oct 28 09:45:58.212999 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 88/180] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:45:57 2022] POST /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7/action => generated 32 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 28 09:45:58.218691 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-34b3f5ec-471c-44c8-8e98-adae3e286822 req-c1737671-190d-4c0d-bb0a-29a360407458 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:58.219371 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-34b3f5ec-471c-44c8-8e98-adae3e286822 req-c1737671-190d-4c0d-bb0a-29a360407458 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:58.220017 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-34b3f5ec-471c-44c8-8e98-adae3e286822 req-c1737671-190d-4c0d-bb0a-29a360407458 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:58.226969 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d6b62ae3-adee-4f77-8329-07303aa550a0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:58.227671 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d6b62ae3-adee-4f77-8329-07303aa550a0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:58.233150 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-34b3f5ec-471c-44c8-8e98-adae3e286822 req-c1737671-190d-4c0d-bb0a-29a360407458 service nova] Creating event network-vif-plugged:e1425374-6966-45a1-bf96-5df1dadc813c for instance b981eca6-4d6f-4e48-bf62-016833f2885d on hv-cinder-zed-3 Oct 28 09:45:58.234351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6b62ae3-adee-4f77-8329-07303aa550a0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:58.234998 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6b62ae3-adee-4f77-8329-07303aa550a0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:58.235719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6b62ae3-adee-4f77-8329-07303aa550a0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:58.236664 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6b62ae3-adee-4f77-8329-07303aa550a0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:58.237271 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6b62ae3-adee-4f77-8329-07303aa550a0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:58.237850 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6b62ae3-adee-4f77-8329-07303aa550a0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:58.239799 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-34b3f5ec-471c-44c8-8e98-adae3e286822 req-c1737671-190d-4c0d-bb0a-29a360407458 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041550 Oct 28 09:45:58.244911 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 93/181] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:45:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:45:58.430902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d6b62ae3-adee-4f77-8329-07303aa550a0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:58.437949 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d6b62ae3-adee-4f77-8329-07303aa550a0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.219173 Oct 28 09:45:58.438536 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 89/182] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:58 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:58.468969 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-40b2dd97-bbcb-4068-b5a0-736a9f506f52 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:58.469720 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-40b2dd97-bbcb-4068-b5a0-736a9f506f52 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:58.476185 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40b2dd97-bbcb-4068-b5a0-736a9f506f52 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:58.476576 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40b2dd97-bbcb-4068-b5a0-736a9f506f52 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:58.477005 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40b2dd97-bbcb-4068-b5a0-736a9f506f52 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:58.477694 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40b2dd97-bbcb-4068-b5a0-736a9f506f52 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:58.478137 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40b2dd97-bbcb-4068-b5a0-736a9f506f52 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:58.478604 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40b2dd97-bbcb-4068-b5a0-736a9f506f52 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:58.496293 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e3f11233-1aaa-467d-9efb-7e04792821c9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:58.498006 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e3f11233-1aaa-467d-9efb-7e04792821c9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:58.505447 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e3f11233-1aaa-467d-9efb-7e04792821c9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:58.505932 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e3f11233-1aaa-467d-9efb-7e04792821c9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:58.506535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e3f11233-1aaa-467d-9efb-7e04792821c9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:58.507217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e3f11233-1aaa-467d-9efb-7e04792821c9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:58.507828 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e3f11233-1aaa-467d-9efb-7e04792821c9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:58.508241 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e3f11233-1aaa-467d-9efb-7e04792821c9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:58.709510 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-40b2dd97-bbcb-4068-b5a0-736a9f506f52 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:58.716096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e3f11233-1aaa-467d-9efb-7e04792821c9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:58.718849 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-40b2dd97-bbcb-4068-b5a0-736a9f506f52 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:58.723298 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-40b2dd97-bbcb-4068-b5a0-736a9f506f52 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:58.724662 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e3f11233-1aaa-467d-9efb-7e04792821c9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.237619 Oct 28 09:45:58.725495 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-40b2dd97-bbcb-4068-b5a0-736a9f506f52 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.260454 Oct 28 09:45:58.725587 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 90/183] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:58 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:58.725875 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 94/184] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:58 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:45:59.199708 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c41a6e52-0cd7-4354-a12d-f943563ba7da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:59.200311 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c41a6e52-0cd7-4354-a12d-f943563ba7da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:59.206216 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c41a6e52-0cd7-4354-a12d-f943563ba7da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:59.206731 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c41a6e52-0cd7-4354-a12d-f943563ba7da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:59.207155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c41a6e52-0cd7-4354-a12d-f943563ba7da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:59.207828 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c41a6e52-0cd7-4354-a12d-f943563ba7da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:59.208765 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c41a6e52-0cd7-4354-a12d-f943563ba7da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:59.209185 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c41a6e52-0cd7-4354-a12d-f943563ba7da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:59.342732 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c41a6e52-0cd7-4354-a12d-f943563ba7da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:59.350373 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c41a6e52-0cd7-4354-a12d-f943563ba7da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.154486 Oct 28 09:45:59.351002 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 91/185] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:59 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:59.461260 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-70bbb2fe-2531-4cd1-a44c-88a79cddc245 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:59.461871 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-70bbb2fe-2531-4cd1-a44c-88a79cddc245 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:59.467852 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-70bbb2fe-2531-4cd1-a44c-88a79cddc245 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:59.468354 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-70bbb2fe-2531-4cd1-a44c-88a79cddc245 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:59.468806 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-70bbb2fe-2531-4cd1-a44c-88a79cddc245 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:59.469491 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-70bbb2fe-2531-4cd1-a44c-88a79cddc245 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:59.469935 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-70bbb2fe-2531-4cd1-a44c-88a79cddc245 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:59.470353 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-70bbb2fe-2531-4cd1-a44c-88a79cddc245 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:59.599512 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-70bbb2fe-2531-4cd1-a44c-88a79cddc245 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:59.607215 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-70bbb2fe-2531-4cd1-a44c-88a79cddc245 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.150165 Oct 28 09:45:59.607785 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 95/186] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:59 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:45:59.748989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1e6470dc-60a7-468b-a2db-6c4f6ec84837 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:59.749586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1e6470dc-60a7-468b-a2db-6c4f6ec84837 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:59.750325 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-7e2829c8-f035-4efc-af2c-80a41d60dff4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:45:59.750959 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-7e2829c8-f035-4efc-af2c-80a41d60dff4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:45:59.756036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1e6470dc-60a7-468b-a2db-6c4f6ec84837 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:59.756502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1e6470dc-60a7-468b-a2db-6c4f6ec84837 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:59.756919 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1e6470dc-60a7-468b-a2db-6c4f6ec84837 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:59.757247 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7e2829c8-f035-4efc-af2c-80a41d60dff4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:59.757618 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1e6470dc-60a7-468b-a2db-6c4f6ec84837 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:59.757721 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7e2829c8-f035-4efc-af2c-80a41d60dff4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:59.758055 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1e6470dc-60a7-468b-a2db-6c4f6ec84837 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:59.758180 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7e2829c8-f035-4efc-af2c-80a41d60dff4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:59.758497 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1e6470dc-60a7-468b-a2db-6c4f6ec84837 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:59.758831 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7e2829c8-f035-4efc-af2c-80a41d60dff4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:45:59.759277 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7e2829c8-f035-4efc-af2c-80a41d60dff4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:45:59.759693 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7e2829c8-f035-4efc-af2c-80a41d60dff4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:45:59.957509 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7e2829c8-f035-4efc-af2c-80a41d60dff4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:59.960872 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1e6470dc-60a7-468b-a2db-6c4f6ec84837 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:59.966814 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7e2829c8-f035-4efc-af2c-80a41d60dff4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:59.968316 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7e2829c8-f035-4efc-af2c-80a41d60dff4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:45:59.969423 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-7e2829c8-f035-4efc-af2c-80a41d60dff4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.223044 Oct 28 09:45:59.969983 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 96/187] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:45:59 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:45:59.971800 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1e6470dc-60a7-468b-a2db-6c4f6ec84837 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.226820 Oct 28 09:45:59.972363 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 92/188] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:45:59 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:00.373651 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c41caf54-a024-4b50-b02e-36de7c451443 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:00.374249 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c41caf54-a024-4b50-b02e-36de7c451443 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:00.380761 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c41caf54-a024-4b50-b02e-36de7c451443 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:00.381243 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c41caf54-a024-4b50-b02e-36de7c451443 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:00.381671 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c41caf54-a024-4b50-b02e-36de7c451443 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:00.382349 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c41caf54-a024-4b50-b02e-36de7c451443 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:00.382827 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c41caf54-a024-4b50-b02e-36de7c451443 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:00.383245 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c41caf54-a024-4b50-b02e-36de7c451443 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:00.509732 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c41caf54-a024-4b50-b02e-36de7c451443 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:00.517446 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c41caf54-a024-4b50-b02e-36de7c451443 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.147601 Oct 28 09:46:00.518018 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 97/189] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:00 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:00.629264 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2ee37016-f52c-4c71-bd18-f7cc35b8a005 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:00.629866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2ee37016-f52c-4c71-bd18-f7cc35b8a005 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:00.636322 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ee37016-f52c-4c71-bd18-f7cc35b8a005 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:00.636794 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ee37016-f52c-4c71-bd18-f7cc35b8a005 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:00.637222 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ee37016-f52c-4c71-bd18-f7cc35b8a005 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:00.637898 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ee37016-f52c-4c71-bd18-f7cc35b8a005 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:00.638345 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ee37016-f52c-4c71-bd18-f7cc35b8a005 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:00.638779 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ee37016-f52c-4c71-bd18-f7cc35b8a005 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:00.768458 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2ee37016-f52c-4c71-bd18-f7cc35b8a005 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:00.777716 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2ee37016-f52c-4c71-bd18-f7cc35b8a005 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.151842 Oct 28 09:46:00.778331 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 93/190] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:00 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:00.994636 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-7fbf08fb-8c50-40d1-89b5-01c8b7430463 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:00.995274 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-7fbf08fb-8c50-40d1-89b5-01c8b7430463 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:01.004646 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fbf08fb-8c50-40d1-89b5-01c8b7430463 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:01.005130 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fbf08fb-8c50-40d1-89b5-01c8b7430463 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:01.005561 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fbf08fb-8c50-40d1-89b5-01c8b7430463 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:01.006279 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fbf08fb-8c50-40d1-89b5-01c8b7430463 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:01.006760 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fbf08fb-8c50-40d1-89b5-01c8b7430463 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:01.007177 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fbf08fb-8c50-40d1-89b5-01c8b7430463 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:01.012649 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-61454394-0007-4897-8e27-6cc4f0fd156d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:01.013248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-61454394-0007-4897-8e27-6cc4f0fd156d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:01.019248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-61454394-0007-4897-8e27-6cc4f0fd156d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:01.019729 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-61454394-0007-4897-8e27-6cc4f0fd156d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:01.020155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-61454394-0007-4897-8e27-6cc4f0fd156d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:01.021339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-61454394-0007-4897-8e27-6cc4f0fd156d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:01.021780 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-61454394-0007-4897-8e27-6cc4f0fd156d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:01.022192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-61454394-0007-4897-8e27-6cc4f0fd156d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:01.206003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7fbf08fb-8c50-40d1-89b5-01c8b7430463 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:01.210552 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-61454394-0007-4897-8e27-6cc4f0fd156d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:01.216772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7fbf08fb-8c50-40d1-89b5-01c8b7430463 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:01.218853 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7fbf08fb-8c50-40d1-89b5-01c8b7430463 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:01.219954 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-7fbf08fb-8c50-40d1-89b5-01c8b7430463 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.229319 Oct 28 09:46:01.220951 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 98/191] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:46:00 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:46:01.234780 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-61454394-0007-4897-8e27-6cc4f0fd156d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.222005 Oct 28 09:46:01.235401 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 94/192] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:00 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:01.541560 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-22a9561e-45c7-497f-967c-ca97256ec777 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:01.542168 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-22a9561e-45c7-497f-967c-ca97256ec777 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:01.548625 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-22a9561e-45c7-497f-967c-ca97256ec777 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:01.549106 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-22a9561e-45c7-497f-967c-ca97256ec777 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:01.549536 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-22a9561e-45c7-497f-967c-ca97256ec777 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:01.550220 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-22a9561e-45c7-497f-967c-ca97256ec777 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:01.550692 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-22a9561e-45c7-497f-967c-ca97256ec777 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:01.551104 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-22a9561e-45c7-497f-967c-ca97256ec777 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:01.613811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f215ade5-24bc-4996-b015-2a87885f0945 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a13c640e-473a-403c-bae6-410e7cd63735"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:46:01.616283 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f215ade5-24bc-4996-b015-2a87885f0945 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:01.622437 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f215ade5-24bc-4996-b015-2a87885f0945 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:01.622929 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f215ade5-24bc-4996-b015-2a87885f0945 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:01.623359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f215ade5-24bc-4996-b015-2a87885f0945 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:01.624047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f215ade5-24bc-4996-b015-2a87885f0945 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:01.624492 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f215ade5-24bc-4996-b015-2a87885f0945 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:01.625016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f215ade5-24bc-4996-b015-2a87885f0945 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:01.721986 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-22a9561e-45c7-497f-967c-ca97256ec777 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:01.731067 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-22a9561e-45c7-497f-967c-ca97256ec777 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.194147 Oct 28 09:46:01.731715 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 99/193] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:01 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:01.801052 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-61ba24f4-7770-4838-a6d9-5ac7d4a63806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:01.801662 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-61ba24f4-7770-4838-a6d9-5ac7d4a63806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:01.808324 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61ba24f4-7770-4838-a6d9-5ac7d4a63806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:01.808903 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61ba24f4-7770-4838-a6d9-5ac7d4a63806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:01.809512 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61ba24f4-7770-4838-a6d9-5ac7d4a63806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:01.810193 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61ba24f4-7770-4838-a6d9-5ac7d4a63806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:01.810669 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61ba24f4-7770-4838-a6d9-5ac7d4a63806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:01.811150 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61ba24f4-7770-4838-a6d9-5ac7d4a63806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:01.949600 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-61ba24f4-7770-4838-a6d9-5ac7d4a63806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:01.957456 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-61ba24f4-7770-4838-a6d9-5ac7d4a63806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.160377 Oct 28 09:46:01.970415 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 100/194] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:01 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:02.260906 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-fd1f1847-04b2-48da-b738-6e247acf9be1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:02.261698 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-fd1f1847-04b2-48da-b738-6e247acf9be1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:02.268174 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fd1f1847-04b2-48da-b738-6e247acf9be1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:02.268736 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fd1f1847-04b2-48da-b738-6e247acf9be1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:02.269147 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fd1f1847-04b2-48da-b738-6e247acf9be1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:02.269877 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fd1f1847-04b2-48da-b738-6e247acf9be1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:02.270328 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fd1f1847-04b2-48da-b738-6e247acf9be1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:02.270779 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fd1f1847-04b2-48da-b738-6e247acf9be1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:02.396891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-fd1f1847-04b2-48da-b738-6e247acf9be1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:02.404947 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-fd1f1847-04b2-48da-b738-6e247acf9be1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:02.406372 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-fd1f1847-04b2-48da-b738-6e247acf9be1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:02.407480 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-fd1f1847-04b2-48da-b738-6e247acf9be1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.153778 Oct 28 09:46:02.408032 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 101/195] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:46:02 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:46:02.413779 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e9a78a0f-c66f-46e5-84a1-3f4231b98ec6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:02.414509 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e9a78a0f-c66f-46e5-84a1-3f4231b98ec6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:02.420490 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e9a78a0f-c66f-46e5-84a1-3f4231b98ec6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:02.420956 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e9a78a0f-c66f-46e5-84a1-3f4231b98ec6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:02.421379 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e9a78a0f-c66f-46e5-84a1-3f4231b98ec6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:02.422055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e9a78a0f-c66f-46e5-84a1-3f4231b98ec6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:02.422519 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e9a78a0f-c66f-46e5-84a1-3f4231b98ec6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:02.423022 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e9a78a0f-c66f-46e5-84a1-3f4231b98ec6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:02.550178 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e9a78a0f-c66f-46e5-84a1-3f4231b98ec6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:02.557885 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e9a78a0f-c66f-46e5-84a1-3f4231b98ec6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.148138 Oct 28 09:46:02.558469 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 102/196] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:02 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:02.754579 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-51564501-b6d9-4853-8cea-13dd3554b558 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:02.755869 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-51564501-b6d9-4853-8cea-13dd3554b558 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:02.762088 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-51564501-b6d9-4853-8cea-13dd3554b558 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:02.762630 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-51564501-b6d9-4853-8cea-13dd3554b558 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:02.763066 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-51564501-b6d9-4853-8cea-13dd3554b558 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:02.763744 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-51564501-b6d9-4853-8cea-13dd3554b558 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:02.764194 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-51564501-b6d9-4853-8cea-13dd3554b558 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:02.764706 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-51564501-b6d9-4853-8cea-13dd3554b558 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:02.890128 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-51564501-b6d9-4853-8cea-13dd3554b558 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:02.893780 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:02.900199 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-51564501-b6d9-4853-8cea-13dd3554b558 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.149643 Oct 28 09:46:02.900763 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 103/197] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:02 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:02.980815 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-02c48302-75fb-4e73-9aa8-fb66ccb1812b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:02.981540 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-02c48302-75fb-4e73-9aa8-fb66ccb1812b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:02.987472 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-02c48302-75fb-4e73-9aa8-fb66ccb1812b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:02.987951 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-02c48302-75fb-4e73-9aa8-fb66ccb1812b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:02.988376 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-02c48302-75fb-4e73-9aa8-fb66ccb1812b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:02.989052 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-02c48302-75fb-4e73-9aa8-fb66ccb1812b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:02.990000 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-02c48302-75fb-4e73-9aa8-fb66ccb1812b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:02.990435 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-02c48302-75fb-4e73-9aa8-fb66ccb1812b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:03.118011 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-02c48302-75fb-4e73-9aa8-fb66ccb1812b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:03.125750 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-02c48302-75fb-4e73-9aa8-fb66ccb1812b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.148976 Oct 28 09:46:03.126310 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 104/198] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:02 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:03.433411 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-40449f54-3098-43ea-826e-f4722863e685 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:03.434112 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-40449f54-3098-43ea-826e-f4722863e685 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:03.440159 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40449f54-3098-43ea-826e-f4722863e685 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:03.440640 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40449f54-3098-43ea-826e-f4722863e685 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:03.441065 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40449f54-3098-43ea-826e-f4722863e685 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:03.441741 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40449f54-3098-43ea-826e-f4722863e685 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:03.442181 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40449f54-3098-43ea-826e-f4722863e685 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:03.442644 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40449f54-3098-43ea-826e-f4722863e685 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:03.757384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-40449f54-3098-43ea-826e-f4722863e685 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:03.767025 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-40449f54-3098-43ea-826e-f4722863e685 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:03.768640 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-40449f54-3098-43ea-826e-f4722863e685 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:03.769793 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-40449f54-3098-43ea-826e-f4722863e685 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.340260 Oct 28 09:46:03.786608 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 105/199] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:46:03 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:46:03.786608 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-fc0854da-1345-4ad5-8024-a961a7e02d7b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:03.786608 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-fc0854da-1345-4ad5-8024-a961a7e02d7b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:03.786608 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fc0854da-1345-4ad5-8024-a961a7e02d7b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:03.787196 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fc0854da-1345-4ad5-8024-a961a7e02d7b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:03.787196 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fc0854da-1345-4ad5-8024-a961a7e02d7b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:03.787196 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fc0854da-1345-4ad5-8024-a961a7e02d7b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:03.787196 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fc0854da-1345-4ad5-8024-a961a7e02d7b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:03.787489 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fc0854da-1345-4ad5-8024-a961a7e02d7b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:03.908690 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-fc0854da-1345-4ad5-8024-a961a7e02d7b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:03.917082 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-fc0854da-1345-4ad5-8024-a961a7e02d7b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.145708 Oct 28 09:46:03.917758 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 106/200] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:03 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:03.922800 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-39c2fdd9-fb49-4944-b842-d53566e7a79a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:03.923531 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-39c2fdd9-fb49-4944-b842-d53566e7a79a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:03.929313 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-39c2fdd9-fb49-4944-b842-d53566e7a79a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:03.929786 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-39c2fdd9-fb49-4944-b842-d53566e7a79a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:03.930212 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-39c2fdd9-fb49-4944-b842-d53566e7a79a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:03.930947 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-39c2fdd9-fb49-4944-b842-d53566e7a79a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:03.931432 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-39c2fdd9-fb49-4944-b842-d53566e7a79a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:03.931798 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-39c2fdd9-fb49-4944-b842-d53566e7a79a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:04.063723 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-39c2fdd9-fb49-4944-b842-d53566e7a79a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:04.071451 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-39c2fdd9-fb49-4944-b842-d53566e7a79a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.152111 Oct 28 09:46:04.072009 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 107/201] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:03 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:04.150008 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-796886f5-3d90-4854-bb85-da8618d87144 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:04.150641 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-796886f5-3d90-4854-bb85-da8618d87144 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:04.156629 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-796886f5-3d90-4854-bb85-da8618d87144 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:04.157108 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-796886f5-3d90-4854-bb85-da8618d87144 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:04.157534 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-796886f5-3d90-4854-bb85-da8618d87144 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:04.158217 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-796886f5-3d90-4854-bb85-da8618d87144 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:04.158698 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-796886f5-3d90-4854-bb85-da8618d87144 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:04.159114 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-796886f5-3d90-4854-bb85-da8618d87144 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:04.286910 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-796886f5-3d90-4854-bb85-da8618d87144 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:04.294808 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-796886f5-3d90-4854-bb85-da8618d87144 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.148822 Oct 28 09:46:04.295372 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 108/202] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:04 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:04.795933 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-4589380a-d375-4d08-a829-49c96ee3ca86 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:04.796559 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-4589380a-d375-4d08-a829-49c96ee3ca86 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:04.803122 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4589380a-d375-4d08-a829-49c96ee3ca86 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:04.803606 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4589380a-d375-4d08-a829-49c96ee3ca86 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:04.804036 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4589380a-d375-4d08-a829-49c96ee3ca86 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:04.804734 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4589380a-d375-4d08-a829-49c96ee3ca86 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:04.805178 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4589380a-d375-4d08-a829-49c96ee3ca86 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:04.805595 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4589380a-d375-4d08-a829-49c96ee3ca86 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:05.087807 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-4589380a-d375-4d08-a829-49c96ee3ca86 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:05.096980 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-4589380a-d375-4d08-a829-49c96ee3ca86 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:05.099832 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-4589380a-d375-4d08-a829-49c96ee3ca86 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:05.101222 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-4589380a-d375-4d08-a829-49c96ee3ca86 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.309120 Oct 28 09:46:05.119621 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 109/203] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:46:04 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:46:05.126558 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b4ad3e3a-cb49-46f8-940a-e7f77bcb2e56 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:05.127430 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b4ad3e3a-cb49-46f8-940a-e7f77bcb2e56 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:05.134462 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b4ad3e3a-cb49-46f8-940a-e7f77bcb2e56 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:05.135191 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b4ad3e3a-cb49-46f8-940a-e7f77bcb2e56 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:05.135780 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b4ad3e3a-cb49-46f8-940a-e7f77bcb2e56 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:05.136770 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b4ad3e3a-cb49-46f8-940a-e7f77bcb2e56 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:05.137368 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b4ad3e3a-cb49-46f8-940a-e7f77bcb2e56 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:05.137922 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b4ad3e3a-cb49-46f8-940a-e7f77bcb2e56 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:05.470447 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b4ad3e3a-cb49-46f8-940a-e7f77bcb2e56 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:05.472177 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f215ade5-24bc-4996-b015-2a87885f0945 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "POST /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.860501 Oct 28 09:46:05.476146 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b4ad3e3a-cb49-46f8-940a-e7f77bcb2e56 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.354670 Oct 28 09:46:05.476497 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 95/204] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:46:01 2022] POST /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc/os-volume_attachments => generated 194 bytes in 3867 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:46:05.477883 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 110/205] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:05 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:05.482858 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-42458c39-06e2-447f-aac1-150f08da9ea4 req-b7f64356-4761-47d2-9845-2dc40c2b02d5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "82790f9d-85c2-4205-b9ff-290df00c2edf", "name": "network-vif-plugged", "status": "completed", "tag": "392bf558-627c-4e33-82ed-38a82396d7b0"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:46:05.488244 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1a4d296a-b3c5-4f34-adbb-73b5d12635ac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:05.490625 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1a4d296a-b3c5-4f34-adbb-73b5d12635ac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:05.494438 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-42458c39-06e2-447f-aac1-150f08da9ea4 req-b7f64356-4761-47d2-9845-2dc40c2b02d5 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:05.495089 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-42458c39-06e2-447f-aac1-150f08da9ea4 req-b7f64356-4761-47d2-9845-2dc40c2b02d5 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:05.495663 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-42458c39-06e2-447f-aac1-150f08da9ea4 req-b7f64356-4761-47d2-9845-2dc40c2b02d5 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:05.503390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1a4d296a-b3c5-4f34-adbb-73b5d12635ac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:05.504252 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1a4d296a-b3c5-4f34-adbb-73b5d12635ac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:05.504791 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1a4d296a-b3c5-4f34-adbb-73b5d12635ac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:05.505783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1a4d296a-b3c5-4f34-adbb-73b5d12635ac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:05.506661 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1a4d296a-b3c5-4f34-adbb-73b5d12635ac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:05.507460 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1a4d296a-b3c5-4f34-adbb-73b5d12635ac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:05.509205 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-42458c39-06e2-447f-aac1-150f08da9ea4 req-b7f64356-4761-47d2-9845-2dc40c2b02d5 service nova] Creating event network-vif-plugged:392bf558-627c-4e33-82ed-38a82396d7b0 for instance 82790f9d-85c2-4205-b9ff-290df00c2edf on hv-cinder-zed-3 Oct 28 09:46:05.514531 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-42458c39-06e2-447f-aac1-150f08da9ea4 req-b7f64356-4761-47d2-9845-2dc40c2b02d5 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035279 Oct 28 09:46:05.515279 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 111/206] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:46:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:46:05.521621 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d3c5d918-3315-426b-bd5a-cc32d5c6d7ca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:05.522516 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d3c5d918-3315-426b-bd5a-cc32d5c6d7ca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:05.529691 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3c5d918-3315-426b-bd5a-cc32d5c6d7ca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:05.530744 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3c5d918-3315-426b-bd5a-cc32d5c6d7ca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:05.531330 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3c5d918-3315-426b-bd5a-cc32d5c6d7ca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:05.532200 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3c5d918-3315-426b-bd5a-cc32d5c6d7ca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:05.532915 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3c5d918-3315-426b-bd5a-cc32d5c6d7ca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:05.533338 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3c5d918-3315-426b-bd5a-cc32d5c6d7ca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:05.745263 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1a4d296a-b3c5-4f34-adbb-73b5d12635ac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:05.749991 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:05.760921 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1a4d296a-b3c5-4f34-adbb-73b5d12635ac tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.282012 Oct 28 09:46:05.762092 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 96/207] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:05 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:05.808196 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d3c5d918-3315-426b-bd5a-cc32d5c6d7ca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:05.817086 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d3c5d918-3315-426b-bd5a-cc32d5c6d7ca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.300312 Oct 28 09:46:05.818529 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 112/208] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:05 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:06.126358 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-349b2419-4206-4c47-a952-0d056a5b6581 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:06.127147 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-349b2419-4206-4c47-a952-0d056a5b6581 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:06.134123 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-349b2419-4206-4c47-a952-0d056a5b6581 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:06.135028 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-349b2419-4206-4c47-a952-0d056a5b6581 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:06.135531 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-349b2419-4206-4c47-a952-0d056a5b6581 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:06.136298 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-349b2419-4206-4c47-a952-0d056a5b6581 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:06.136856 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-349b2419-4206-4c47-a952-0d056a5b6581 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:06.137351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-349b2419-4206-4c47-a952-0d056a5b6581 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:06.303491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-349b2419-4206-4c47-a952-0d056a5b6581 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:06.312155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-349b2419-4206-4c47-a952-0d056a5b6581 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:06.313574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-349b2419-4206-4c47-a952-0d056a5b6581 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:06.314714 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-349b2419-4206-4c47-a952-0d056a5b6581 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1407 microversion: 2.20 time: 0.192925 Oct 28 09:46:06.315256 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 97/209] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:46:06 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1407 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:46:06.501137 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-247e1caf-bdc6-44ba-a821-a2b6707b0c42 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:06.501823 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-247e1caf-bdc6-44ba-a821-a2b6707b0c42 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:06.508640 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-247e1caf-bdc6-44ba-a821-a2b6707b0c42 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:06.509219 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-247e1caf-bdc6-44ba-a821-a2b6707b0c42 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:06.509722 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-247e1caf-bdc6-44ba-a821-a2b6707b0c42 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:06.510507 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-247e1caf-bdc6-44ba-a821-a2b6707b0c42 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:06.510952 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-247e1caf-bdc6-44ba-a821-a2b6707b0c42 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:06.511367 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-247e1caf-bdc6-44ba-a821-a2b6707b0c42 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:06.661324 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-247e1caf-bdc6-44ba-a821-a2b6707b0c42 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:06.671147 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-247e1caf-bdc6-44ba-a821-a2b6707b0c42 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.173798 Oct 28 09:46:06.675480 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 113/210] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:06 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:06.786728 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ab4589a3-4190-4b9f-85dc-750509d52b46 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:06.787371 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ab4589a3-4190-4b9f-85dc-750509d52b46 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:06.793557 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab4589a3-4190-4b9f-85dc-750509d52b46 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:06.794191 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab4589a3-4190-4b9f-85dc-750509d52b46 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:06.794806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab4589a3-4190-4b9f-85dc-750509d52b46 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:06.795495 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab4589a3-4190-4b9f-85dc-750509d52b46 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:06.795965 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab4589a3-4190-4b9f-85dc-750509d52b46 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:06.796380 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab4589a3-4190-4b9f-85dc-750509d52b46 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:06.841030 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-f12dd09a-9821-4c7f-9857-1cde4c990f57 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:06.841631 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-f12dd09a-9821-4c7f-9857-1cde4c990f57 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:06.850533 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f12dd09a-9821-4c7f-9857-1cde4c990f57 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:06.851254 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f12dd09a-9821-4c7f-9857-1cde4c990f57 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:06.851850 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f12dd09a-9821-4c7f-9857-1cde4c990f57 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:06.852725 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f12dd09a-9821-4c7f-9857-1cde4c990f57 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:06.853313 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f12dd09a-9821-4c7f-9857-1cde4c990f57 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:06.853880 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f12dd09a-9821-4c7f-9857-1cde4c990f57 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:07.032697 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ab4589a3-4190-4b9f-85dc-750509d52b46 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:07.041238 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ab4589a3-4190-4b9f-85dc-750509d52b46 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1363 microversion: 2.1 time: 0.258218 Oct 28 09:46:07.042087 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 98/211] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:06 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1363 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:07.073198 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-f12dd09a-9821-4c7f-9857-1cde4c990f57 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:07.087799 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-f12dd09a-9821-4c7f-9857-1cde4c990f57 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.250071 Oct 28 09:46:07.088732 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 114/212] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:06 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:07.341193 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a9553bbb-f98f-4537-a598-3fd5b0d26182 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:07.342652 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a9553bbb-f98f-4537-a598-3fd5b0d26182 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:07.351025 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a9553bbb-f98f-4537-a598-3fd5b0d26182 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:07.351616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a9553bbb-f98f-4537-a598-3fd5b0d26182 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:07.353113 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a9553bbb-f98f-4537-a598-3fd5b0d26182 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:07.353961 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a9553bbb-f98f-4537-a598-3fd5b0d26182 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:07.354441 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a9553bbb-f98f-4537-a598-3fd5b0d26182 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:07.354899 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a9553bbb-f98f-4537-a598-3fd5b0d26182 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:07.545039 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a9553bbb-f98f-4537-a598-3fd5b0d26182 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:07.556683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a9553bbb-f98f-4537-a598-3fd5b0d26182 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:07.558541 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a9553bbb-f98f-4537-a598-3fd5b0d26182 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:07.560518 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a9553bbb-f98f-4537-a598-3fd5b0d26182 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1598 microversion: 2.20 time: 0.224391 Oct 28 09:46:07.561818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 99/213] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:46:07 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1598 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:46:07.588429 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-a64dd272-e633-4152-a454-26c22d520b99 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:07.588944 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-a64dd272-e633-4152-a454-26c22d520b99 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:07.602249 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a64dd272-e633-4152-a454-26c22d520b99 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:07.602786 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a64dd272-e633-4152-a454-26c22d520b99 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:07.603238 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a64dd272-e633-4152-a454-26c22d520b99 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:07.604260 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a64dd272-e633-4152-a454-26c22d520b99 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:07.616984 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a64dd272-e633-4152-a454-26c22d520b99 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:07.616984 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a64dd272-e633-4152-a454-26c22d520b99 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:07.699227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3f857154-499d-402d-af97-7c4d2a8df132 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:07.700210 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3f857154-499d-402d-af97-7c4d2a8df132 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:07.713133 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3f857154-499d-402d-af97-7c4d2a8df132 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:07.713641 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3f857154-499d-402d-af97-7c4d2a8df132 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:07.714073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3f857154-499d-402d-af97-7c4d2a8df132 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:07.714807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3f857154-499d-402d-af97-7c4d2a8df132 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:07.715249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3f857154-499d-402d-af97-7c4d2a8df132 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:07.715676 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3f857154-499d-402d-af97-7c4d2a8df132 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:08.138726 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-a64dd272-e633-4152-a454-26c22d520b99 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.556325 Oct 28 09:46:08.139893 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 115/214] 10.222.0.25 () {60 vars in 1316 bytes} [Fri Oct 28 09:46:07 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-interface => generated 284 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 28 09:46:08.146037 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d4977e0f-77a5-4fda-9632-bae3878192fc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:08.147041 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d4977e0f-77a5-4fda-9632-bae3878192fc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:08.154657 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d4977e0f-77a5-4fda-9632-bae3878192fc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:08.155532 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d4977e0f-77a5-4fda-9632-bae3878192fc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:08.156276 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d4977e0f-77a5-4fda-9632-bae3878192fc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:08.157598 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d4977e0f-77a5-4fda-9632-bae3878192fc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:08.158544 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d4977e0f-77a5-4fda-9632-bae3878192fc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:08.159177 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d4977e0f-77a5-4fda-9632-bae3878192fc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:08.333737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3f857154-499d-402d-af97-7c4d2a8df132 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:08.342160 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3f857154-499d-402d-af97-7c4d2a8df132 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.646895 Oct 28 09:46:08.362332 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 100/215] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:07 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 668 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:08.371753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f1041100-f984-4e66-9bfd-a3c19cfa6ee5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:08.373892 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f1041100-f984-4e66-9bfd-a3c19cfa6ee5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:08.386663 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1041100-f984-4e66-9bfd-a3c19cfa6ee5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:08.387254 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1041100-f984-4e66-9bfd-a3c19cfa6ee5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:08.387696 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1041100-f984-4e66-9bfd-a3c19cfa6ee5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:08.388417 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1041100-f984-4e66-9bfd-a3c19cfa6ee5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:08.388867 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1041100-f984-4e66-9bfd-a3c19cfa6ee5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:08.389400 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1041100-f984-4e66-9bfd-a3c19cfa6ee5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:08.443375 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d4977e0f-77a5-4fda-9632-bae3878192fc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:08.453319 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d4977e0f-77a5-4fda-9632-bae3878192fc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1549 microversion: 2.1 time: 0.311987 Oct 28 09:46:08.454783 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 116/216] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:08 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1549 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:08.478490 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-82c643e7-6f9e-45c1-b019-5870bb7d283a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:08.478955 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-82c643e7-6f9e-45c1-b019-5870bb7d283a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:08.486011 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-82c643e7-6f9e-45c1-b019-5870bb7d283a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:08.486525 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-82c643e7-6f9e-45c1-b019-5870bb7d283a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:08.486962 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-82c643e7-6f9e-45c1-b019-5870bb7d283a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:08.487645 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-82c643e7-6f9e-45c1-b019-5870bb7d283a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:08.488642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-82c643e7-6f9e-45c1-b019-5870bb7d283a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:08.488642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-82c643e7-6f9e-45c1-b019-5870bb7d283a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:08.654862 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-82c643e7-6f9e-45c1-b019-5870bb7d283a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.180698 Oct 28 09:46:08.655805 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 117/217] 10.222.0.25 () {58 vars in 1275 bytes} [Fri Oct 28 09:46:08 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/os-interface => generated 284 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:46:08.753742 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f1041100-f984-4e66-9bfd-a3c19cfa6ee5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:08.767677 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f1041100-f984-4e66-9bfd-a3c19cfa6ee5 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.399987 Oct 28 09:46:08.771172 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 101/218] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:08 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:09.366344 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e539e430-be7d-40d2-9a58-e4c565a481c0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:09.367103 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e539e430-be7d-40d2-9a58-e4c565a481c0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:09.377206 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e539e430-be7d-40d2-9a58-e4c565a481c0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:09.377708 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e539e430-be7d-40d2-9a58-e4c565a481c0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:09.379417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e539e430-be7d-40d2-9a58-e4c565a481c0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:09.381311 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e539e430-be7d-40d2-9a58-e4c565a481c0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:09.383002 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e539e430-be7d-40d2-9a58-e4c565a481c0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:09.383663 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e539e430-be7d-40d2-9a58-e4c565a481c0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:09.717229 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e539e430-be7d-40d2-9a58-e4c565a481c0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:09.732774 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e539e430-be7d-40d2-9a58-e4c565a481c0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.370991 Oct 28 09:46:09.733373 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 118/219] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:09 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:09.794942 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e95d939f-8412-4c60-b94a-5e54bfa34dec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:09.796249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e95d939f-8412-4c60-b94a-5e54bfa34dec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:09.805555 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e95d939f-8412-4c60-b94a-5e54bfa34dec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:09.808356 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e95d939f-8412-4c60-b94a-5e54bfa34dec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:09.809150 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e95d939f-8412-4c60-b94a-5e54bfa34dec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:09.809884 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e95d939f-8412-4c60-b94a-5e54bfa34dec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:09.810533 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e95d939f-8412-4c60-b94a-5e54bfa34dec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:09.811104 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e95d939f-8412-4c60-b94a-5e54bfa34dec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:10.127116 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e95d939f-8412-4c60-b94a-5e54bfa34dec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:10.143891 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e95d939f-8412-4c60-b94a-5e54bfa34dec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.352957 Oct 28 09:46:10.148996 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 102/220] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:09 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:10.755624 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5899b6f1-09eb-4077-b61d-a1823776ea65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:10.759708 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5899b6f1-09eb-4077-b61d-a1823776ea65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:10.769355 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5899b6f1-09eb-4077-b61d-a1823776ea65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:10.770156 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5899b6f1-09eb-4077-b61d-a1823776ea65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:10.770342 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5899b6f1-09eb-4077-b61d-a1823776ea65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:10.772209 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5899b6f1-09eb-4077-b61d-a1823776ea65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:10.772765 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5899b6f1-09eb-4077-b61d-a1823776ea65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:10.773247 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5899b6f1-09eb-4077-b61d-a1823776ea65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:11.105197 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5899b6f1-09eb-4077-b61d-a1823776ea65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:11.116318 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5899b6f1-09eb-4077-b61d-a1823776ea65 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.364124 Oct 28 09:46:11.117330 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 119/221] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:10 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:11.173163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f647c7ec-27cd-4ecb-acfc-6dc57c19dc5f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:11.173994 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f647c7ec-27cd-4ecb-acfc-6dc57c19dc5f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:11.182952 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f647c7ec-27cd-4ecb-acfc-6dc57c19dc5f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:11.183814 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f647c7ec-27cd-4ecb-acfc-6dc57c19dc5f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:11.184853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f647c7ec-27cd-4ecb-acfc-6dc57c19dc5f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:11.187373 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f647c7ec-27cd-4ecb-acfc-6dc57c19dc5f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:11.188242 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f647c7ec-27cd-4ecb-acfc-6dc57c19dc5f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:11.188994 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f647c7ec-27cd-4ecb-acfc-6dc57c19dc5f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:11.373511 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f647c7ec-27cd-4ecb-acfc-6dc57c19dc5f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:11.388102 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f647c7ec-27cd-4ecb-acfc-6dc57c19dc5f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.217226 Oct 28 09:46:11.404125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 103/222] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:11 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:11.760355 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-3747ca1e-8550-43c9-b1c9-e3863feda2be req-02db7921-786d-411b-90e3-d584e5b6184a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c716d5fd-f257-4f6a-8f52-18ad59e5c31e", "tag": "c2310b44-82bb-408f-85b5-8a5556b92e25"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:46:11.779613 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-3747ca1e-8550-43c9-b1c9-e3863feda2be req-02db7921-786d-411b-90e3-d584e5b6184a service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:11.780100 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-3747ca1e-8550-43c9-b1c9-e3863feda2be req-02db7921-786d-411b-90e3-d584e5b6184a service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:11.780567 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-3747ca1e-8550-43c9-b1c9-e3863feda2be req-02db7921-786d-411b-90e3-d584e5b6184a service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:11.804506 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-3747ca1e-8550-43c9-b1c9-e3863feda2be req-02db7921-786d-411b-90e3-d584e5b6184a service nova] Creating event network-changed:c2310b44-82bb-408f-85b5-8a5556b92e25 for instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e on hv-cinder-zed-3 Oct 28 09:46:11.817277 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-3747ca1e-8550-43c9-b1c9-e3863feda2be req-02db7921-786d-411b-90e3-d584e5b6184a service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060660 Oct 28 09:46:11.820733 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 120/223] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:46:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:46:12.142127 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-07b6de5e-e215-4846-b5fc-4b457bb42430 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:12.142744 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-07b6de5e-e215-4846-b5fc-4b457bb42430 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:12.156853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-07b6de5e-e215-4846-b5fc-4b457bb42430 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:12.157360 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-07b6de5e-e215-4846-b5fc-4b457bb42430 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:12.163287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-07b6de5e-e215-4846-b5fc-4b457bb42430 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:12.164051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-07b6de5e-e215-4846-b5fc-4b457bb42430 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:12.164529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-07b6de5e-e215-4846-b5fc-4b457bb42430 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:12.175057 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-07b6de5e-e215-4846-b5fc-4b457bb42430 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:12.413349 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-484f9728-79bc-4f19-bfbf-e419d97e1ae1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:12.414210 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-484f9728-79bc-4f19-bfbf-e419d97e1ae1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:12.443328 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-484f9728-79bc-4f19-bfbf-e419d97e1ae1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:12.462627 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-484f9728-79bc-4f19-bfbf-e419d97e1ae1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:12.462627 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-484f9728-79bc-4f19-bfbf-e419d97e1ae1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:12.462627 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-484f9728-79bc-4f19-bfbf-e419d97e1ae1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:12.462627 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-484f9728-79bc-4f19-bfbf-e419d97e1ae1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:12.462943 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-484f9728-79bc-4f19-bfbf-e419d97e1ae1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:12.779684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-07b6de5e-e215-4846-b5fc-4b457bb42430 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:12.795102 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-07b6de5e-e215-4846-b5fc-4b457bb42430 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1366 microversion: 2.1 time: 0.656939 Oct 28 09:46:12.799081 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 104/224] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:12 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1366 bytes in 662 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:12.804741 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e73e9a06-47d2-46ba-b07a-5634f23c7e05 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:12.805488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e73e9a06-47d2-46ba-b07a-5634f23c7e05 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:12.814846 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e73e9a06-47d2-46ba-b07a-5634f23c7e05 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:12.815581 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e73e9a06-47d2-46ba-b07a-5634f23c7e05 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:12.818112 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e73e9a06-47d2-46ba-b07a-5634f23c7e05 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:12.826695 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e73e9a06-47d2-46ba-b07a-5634f23c7e05 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:12.827190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e73e9a06-47d2-46ba-b07a-5634f23c7e05 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:12.827629 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e73e9a06-47d2-46ba-b07a-5634f23c7e05 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:12.912019 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e73e9a06-47d2-46ba-b07a-5634f23c7e05 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Going to try to terminate instance {{(pid=127030) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:46:13.063932 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-484f9728-79bc-4f19-bfbf-e419d97e1ae1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:13.074544 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-484f9728-79bc-4f19-bfbf-e419d97e1ae1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.667557 Oct 28 09:46:13.087404 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 121/225] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:12 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 681 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:13.123680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e73e9a06-47d2-46ba-b07a-5634f23c7e05 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "DELETE /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 204 len: 0 microversion: 2.1 time: 0.322651 Oct 28 09:46:13.125583 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 105/226] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:46:12 2022] DELETE /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 0 bytes in 325 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:46:13.135841 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e122eab8-b297-4da3-85bb-b0774f916513 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:13.137135 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e122eab8-b297-4da3-85bb-b0774f916513 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:13.149934 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e122eab8-b297-4da3-85bb-b0774f916513 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:13.150480 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e122eab8-b297-4da3-85bb-b0774f916513 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:13.151089 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e122eab8-b297-4da3-85bb-b0774f916513 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:13.151963 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e122eab8-b297-4da3-85bb-b0774f916513 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:13.153635 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e122eab8-b297-4da3-85bb-b0774f916513 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:13.155280 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e122eab8-b297-4da3-85bb-b0774f916513 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:13.203577 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:13.806741 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e122eab8-b297-4da3-85bb-b0774f916513 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:13.829304 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-bcc3c0fb-e099-4f58-8598-d8bf27865a02 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:13.831552 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-bcc3c0fb-e099-4f58-8598-d8bf27865a02 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:13.841733 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e122eab8-b297-4da3-85bb-b0774f916513 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1553 microversion: 2.1 time: 0.710117 Oct 28 09:46:13.845370 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 122/227] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:13 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1553 bytes in 716 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:13.848129 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bcc3c0fb-e099-4f58-8598-d8bf27865a02 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:13.849660 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bcc3c0fb-e099-4f58-8598-d8bf27865a02 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:13.850312 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bcc3c0fb-e099-4f58-8598-d8bf27865a02 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:13.851247 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bcc3c0fb-e099-4f58-8598-d8bf27865a02 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:13.851864 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bcc3c0fb-e099-4f58-8598-d8bf27865a02 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:13.852963 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bcc3c0fb-e099-4f58-8598-d8bf27865a02 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:13.885418 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-5cd93044-a74d-4a52-a8b8-87cca9b0afd9 req-f267f823-ae00-4b5e-8514-79cd40128ef2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b981eca6-4d6f-4e48-bf62-016833f2885d", "tag": "e1425374-6966-45a1-bf96-5df1dadc813c"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:46:13.920762 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-5cd93044-a74d-4a52-a8b8-87cca9b0afd9 req-f267f823-ae00-4b5e-8514-79cd40128ef2 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:13.921252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-5cd93044-a74d-4a52-a8b8-87cca9b0afd9 req-f267f823-ae00-4b5e-8514-79cd40128ef2 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:13.921729 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-5cd93044-a74d-4a52-a8b8-87cca9b0afd9 req-f267f823-ae00-4b5e-8514-79cd40128ef2 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:13.961908 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-5cd93044-a74d-4a52-a8b8-87cca9b0afd9 req-f267f823-ae00-4b5e-8514-79cd40128ef2 service nova] Creating event network-changed:e1425374-6966-45a1-bf96-5df1dadc813c for instance b981eca6-4d6f-4e48-bf62-016833f2885d on hv-cinder-zed-3 Oct 28 09:46:13.970792 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-5cd93044-a74d-4a52-a8b8-87cca9b0afd9 req-f267f823-ae00-4b5e-8514-79cd40128ef2 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.092980 Oct 28 09:46:13.971969 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 123/228] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:46:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:46:14.104969 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-29281212-4cb2-44e6-ba6c-7289f62dc897 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:14.112857 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-29281212-4cb2-44e6-ba6c-7289f62dc897 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:14.139765 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-29281212-4cb2-44e6-ba6c-7289f62dc897 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:14.145212 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-29281212-4cb2-44e6-ba6c-7289f62dc897 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:14.146669 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-29281212-4cb2-44e6-ba6c-7289f62dc897 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:14.148384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-29281212-4cb2-44e6-ba6c-7289f62dc897 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:14.149019 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-29281212-4cb2-44e6-ba6c-7289f62dc897 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:14.149630 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-29281212-4cb2-44e6-ba6c-7289f62dc897 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:14.652658 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-bcc3c0fb-e099-4f58-8598-d8bf27865a02 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:14.661733 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-bcc3c0fb-e099-4f58-8598-d8bf27865a02 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1557 microversion: 2.1 time: 0.841601 Oct 28 09:46:14.666961 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 106/229] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:13 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1557 bytes in 848 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:14.673153 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-29281212-4cb2-44e6-ba6c-7289f62dc897 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:14.685479 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-29281212-4cb2-44e6-ba6c-7289f62dc897 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.587742 Oct 28 09:46:14.691526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3911c382-b21c-431b-b314-17163422881a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:14.692100 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3911c382-b21c-431b-b314-17163422881a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:14.694311 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 124/230] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:14 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:14.706573 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3911c382-b21c-431b-b314-17163422881a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:14.707310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3911c382-b21c-431b-b314-17163422881a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:14.707831 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3911c382-b21c-431b-b314-17163422881a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:14.708737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3911c382-b21c-431b-b314-17163422881a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:14.709333 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3911c382-b21c-431b-b314-17163422881a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:14.709831 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3911c382-b21c-431b-b314-17163422881a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:14.859061 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3911c382-b21c-431b-b314-17163422881a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.168684 Oct 28 09:46:14.859689 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 107/231] 10.222.0.25 () {58 vars in 1275 bytes} [Fri Oct 28 09:46:14 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-interface => generated 285 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:46:14.868035 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-17b23078-954d-4324-8793-e00e7cac3d61 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:14.868646 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-17b23078-954d-4324-8793-e00e7cac3d61 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:14.883117 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-17b23078-954d-4324-8793-e00e7cac3d61 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:14.883826 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-17b23078-954d-4324-8793-e00e7cac3d61 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:14.884428 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-17b23078-954d-4324-8793-e00e7cac3d61 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:14.885464 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-17b23078-954d-4324-8793-e00e7cac3d61 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:14.886161 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-17b23078-954d-4324-8793-e00e7cac3d61 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:14.887007 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-17b23078-954d-4324-8793-e00e7cac3d61 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:15.175557 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-17b23078-954d-4324-8793-e00e7cac3d61 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:15.186148 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-17b23078-954d-4324-8793-e00e7cac3d61 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1553 microversion: 2.1 time: 0.319852 Oct 28 09:46:15.189018 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 125/232] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:14 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1553 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:15.722122 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-875f700f-ae86-485d-9b77-42e5e7f385e1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:15.723030 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-875f700f-ae86-485d-9b77-42e5e7f385e1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:15.735299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-875f700f-ae86-485d-9b77-42e5e7f385e1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:15.735794 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-875f700f-ae86-485d-9b77-42e5e7f385e1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:15.740997 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-875f700f-ae86-485d-9b77-42e5e7f385e1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:15.741743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-875f700f-ae86-485d-9b77-42e5e7f385e1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:15.742211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-875f700f-ae86-485d-9b77-42e5e7f385e1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:15.742663 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-875f700f-ae86-485d-9b77-42e5e7f385e1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:16.221380 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-387303de-789c-451c-b91e-2b5a606f3e9b tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:16.229798 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-387303de-789c-451c-b91e-2b5a606f3e9b tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:16.279957 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-387303de-789c-451c-b91e-2b5a606f3e9b tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:16.280535 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-387303de-789c-451c-b91e-2b5a606f3e9b tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:16.280999 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-387303de-789c-451c-b91e-2b5a606f3e9b tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:16.281746 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-387303de-789c-451c-b91e-2b5a606f3e9b tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:16.282206 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-387303de-789c-451c-b91e-2b5a606f3e9b tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:16.282660 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-387303de-789c-451c-b91e-2b5a606f3e9b tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:16.384810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-875f700f-ae86-485d-9b77-42e5e7f385e1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:16.394220 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-875f700f-ae86-485d-9b77-42e5e7f385e1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.675820 Oct 28 09:46:16.398161 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 108/233] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:15 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 680 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:17.084027 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-ce83988d-15f0-4f9c-bbbe-5ada326816aa req-e6f18184-1e6f-4b01-88f5-5a79bfb540a1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "82790f9d-85c2-4205-b9ff-290df00c2edf", "tag": "392bf558-627c-4e33-82ed-38a82396d7b0"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:46:17.096785 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-ce83988d-15f0-4f9c-bbbe-5ada326816aa req-e6f18184-1e6f-4b01-88f5-5a79bfb540a1 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:17.097456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-ce83988d-15f0-4f9c-bbbe-5ada326816aa req-e6f18184-1e6f-4b01-88f5-5a79bfb540a1 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:17.098059 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-ce83988d-15f0-4f9c-bbbe-5ada326816aa req-e6f18184-1e6f-4b01-88f5-5a79bfb540a1 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:17.111769 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-ce83988d-15f0-4f9c-bbbe-5ada326816aa req-e6f18184-1e6f-4b01-88f5-5a79bfb540a1 service nova] Creating event network-changed:392bf558-627c-4e33-82ed-38a82396d7b0 for instance 82790f9d-85c2-4205-b9ff-290df00c2edf on hv-cinder-zed-3 Oct 28 09:46:17.119809 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-ce83988d-15f0-4f9c-bbbe-5ada326816aa req-e6f18184-1e6f-4b01-88f5-5a79bfb540a1 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038381 Oct 28 09:46:17.120930 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 109/234] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:46:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:46:17.244621 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-387303de-789c-451c-b91e-2b5a606f3e9b tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:17.260815 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-387303de-789c-451c-b91e-2b5a606f3e9b tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1553 microversion: 2.1 time: 1.053094 Oct 28 09:46:17.265869 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 126/235] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:16 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1553 bytes in 1059 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:17.431516 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1da68b8c-f30e-40b2-8a84-40054a028ddb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:17.434765 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1da68b8c-f30e-40b2-8a84-40054a028ddb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:17.445052 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1da68b8c-f30e-40b2-8a84-40054a028ddb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:17.446727 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1da68b8c-f30e-40b2-8a84-40054a028ddb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:17.447489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1da68b8c-f30e-40b2-8a84-40054a028ddb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:17.448461 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1da68b8c-f30e-40b2-8a84-40054a028ddb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:17.449136 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1da68b8c-f30e-40b2-8a84-40054a028ddb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:17.449771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1da68b8c-f30e-40b2-8a84-40054a028ddb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:18.217873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1da68b8c-f30e-40b2-8a84-40054a028ddb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:18.237314 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1da68b8c-f30e-40b2-8a84-40054a028ddb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.816886 Oct 28 09:46:18.238223 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 110/236] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:17 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 819 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:18.292040 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-f2e87434-6ac4-45b0-9417-7f9796427b75 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:18.293175 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-f2e87434-6ac4-45b0-9417-7f9796427b75 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:18.303694 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f2e87434-6ac4-45b0-9417-7f9796427b75 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:18.304181 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f2e87434-6ac4-45b0-9417-7f9796427b75 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:18.304820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f2e87434-6ac4-45b0-9417-7f9796427b75 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:18.305305 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f2e87434-6ac4-45b0-9417-7f9796427b75 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:18.307860 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f2e87434-6ac4-45b0-9417-7f9796427b75 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:18.308353 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f2e87434-6ac4-45b0-9417-7f9796427b75 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:18.709851 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-f2e87434-6ac4-45b0-9417-7f9796427b75 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:18.718622 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-f2e87434-6ac4-45b0-9417-7f9796427b75 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1553 microversion: 2.1 time: 0.430568 Oct 28 09:46:18.719808 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 127/237] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:18 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1553 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:18.902315 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-cd774831-af9e-4933-bd65-e83eec493879 req-0c3219ff-fb55-4f8b-9489-c15214c31e6b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "088d0e80-3838-4ada-938b-f26f194d5abc", "name": "network-vif-deleted", "tag": "a903c483-3505-4c5d-9652-f34609414d9d"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:46:18.919001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-cd774831-af9e-4933-bd65-e83eec493879 req-0c3219ff-fb55-4f8b-9489-c15214c31e6b service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:18.919565 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-cd774831-af9e-4933-bd65-e83eec493879 req-0c3219ff-fb55-4f8b-9489-c15214c31e6b service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:18.920085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-cd774831-af9e-4933-bd65-e83eec493879 req-0c3219ff-fb55-4f8b-9489-c15214c31e6b service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:18.939071 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-cd774831-af9e-4933-bd65-e83eec493879 req-0c3219ff-fb55-4f8b-9489-c15214c31e6b service nova] Creating event network-vif-deleted:a903c483-3505-4c5d-9652-f34609414d9d for instance 088d0e80-3838-4ada-938b-f26f194d5abc on hv-cinder-zed-3 Oct 28 09:46:18.952377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-cd774831-af9e-4933-bd65-e83eec493879 req-0c3219ff-fb55-4f8b-9489-c15214c31e6b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054263 Oct 28 09:46:18.953655 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 111/238] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:46:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:46:19.264937 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-839aaa89-a5d6-4133-b011-841e143dc70f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:19.265538 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-839aaa89-a5d6-4133-b011-841e143dc70f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:19.271848 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-839aaa89-a5d6-4133-b011-841e143dc70f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:19.272336 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-839aaa89-a5d6-4133-b011-841e143dc70f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:19.272928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-839aaa89-a5d6-4133-b011-841e143dc70f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:19.273791 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-839aaa89-a5d6-4133-b011-841e143dc70f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:19.274592 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-839aaa89-a5d6-4133-b011-841e143dc70f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:19.275274 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-839aaa89-a5d6-4133-b011-841e143dc70f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:19.437749 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-839aaa89-a5d6-4133-b011-841e143dc70f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:19.451854 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-839aaa89-a5d6-4133-b011-841e143dc70f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.190619 Oct 28 09:46:19.452912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 128/239] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:19 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:19.742542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c018c2ff-f27e-4a06-b59f-4416152ceb5c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:19.743549 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c018c2ff-f27e-4a06-b59f-4416152ceb5c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:19.752283 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c018c2ff-f27e-4a06-b59f-4416152ceb5c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:19.753047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c018c2ff-f27e-4a06-b59f-4416152ceb5c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:19.753675 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c018c2ff-f27e-4a06-b59f-4416152ceb5c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:19.754624 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c018c2ff-f27e-4a06-b59f-4416152ceb5c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:19.755341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c018c2ff-f27e-4a06-b59f-4416152ceb5c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:19.756001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c018c2ff-f27e-4a06-b59f-4416152ceb5c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:19.850838 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c018c2ff-f27e-4a06-b59f-4416152ceb5c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:19.863118 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c018c2ff-f27e-4a06-b59f-4416152ceb5c tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1511 microversion: 2.1 time: 0.125692 Oct 28 09:46:19.864219 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 112/240] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:19 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1511 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:20.478185 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2f609abd-d647-4072-a9cc-5f3db40760d6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:20.478761 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2f609abd-d647-4072-a9cc-5f3db40760d6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:20.482026 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:20.485454 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2f609abd-d647-4072-a9cc-5f3db40760d6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:20.486455 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2f609abd-d647-4072-a9cc-5f3db40760d6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:20.486455 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2f609abd-d647-4072-a9cc-5f3db40760d6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:20.487068 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2f609abd-d647-4072-a9cc-5f3db40760d6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:20.487509 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2f609abd-d647-4072-a9cc-5f3db40760d6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:20.487928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2f609abd-d647-4072-a9cc-5f3db40760d6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:20.768767 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2f609abd-d647-4072-a9cc-5f3db40760d6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:20.790024 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2f609abd-d647-4072-a9cc-5f3db40760d6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.315860 Oct 28 09:46:20.792462 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 129/241] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:20 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:20.892139 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ce1b3627-21e5-4abd-ba0c-cc5c0ba530e5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:20.892749 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ce1b3627-21e5-4abd-ba0c-cc5c0ba530e5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:20.898615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=88,nova_api:UPDATE=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:20.900266 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ce1b3627-21e5-4abd-ba0c-cc5c0ba530e5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:20.900897 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ce1b3627-21e5-4abd-ba0c-cc5c0ba530e5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:20.901476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ce1b3627-21e5-4abd-ba0c-cc5c0ba530e5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:20.902349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ce1b3627-21e5-4abd-ba0c-cc5c0ba530e5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:20.902970 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ce1b3627-21e5-4abd-ba0c-cc5c0ba530e5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:20.903532 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ce1b3627-21e5-4abd-ba0c-cc5c0ba530e5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:21.121397 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:21.498383 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ce1b3627-21e5-4abd-ba0c-cc5c0ba530e5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:21.512590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ce1b3627-21e5-4abd-ba0c-cc5c0ba530e5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1345 microversion: 2.1 time: 0.623969 Oct 28 09:46:21.521994 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 113/242] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:20 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1345 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:21.817054 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2534b971-21da-470c-8097-0c6eadeab9bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:21.817731 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2534b971-21da-470c-8097-0c6eadeab9bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:21.827918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2534b971-21da-470c-8097-0c6eadeab9bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:21.828381 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2534b971-21da-470c-8097-0c6eadeab9bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:21.828814 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2534b971-21da-470c-8097-0c6eadeab9bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:21.829509 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2534b971-21da-470c-8097-0c6eadeab9bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:21.829952 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2534b971-21da-470c-8097-0c6eadeab9bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:21.837476 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2534b971-21da-470c-8097-0c6eadeab9bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:22.121184 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2534b971-21da-470c-8097-0c6eadeab9bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:22.129623 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2534b971-21da-470c-8097-0c6eadeab9bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.316902 Oct 28 09:46:22.136146 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 130/243] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:21 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:22.551124 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:22.551768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:22.589220 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:22.589728 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:22.596022 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:22.597151 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:22.597786 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:22.598369 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:22.671365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:22.672814 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:22.673573 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:22.674671 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lazy-loading 'fault' on Instance uuid 088d0e80-3838-4ada-938b-f26f194d5abc {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:46:22.731651 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e451d264e4964409b09e0b96ae8b983e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41516f10d050497e82056ddd1be5981c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:22.767622 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-dbdad3e9-e3d0-47a0-a394-fbd86ae755ab tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 200 len: 1304 microversion: 2.1 time: 0.224467 Oct 28 09:46:22.769580 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 114/244] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:22 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 1304 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:23.168069 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-dd0680a5-f0fa-4e9f-b80f-a3418f8883a4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:23.168690 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-dd0680a5-f0fa-4e9f-b80f-a3418f8883a4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:23.181015 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-dd0680a5-f0fa-4e9f-b80f-a3418f8883a4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:23.181769 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-dd0680a5-f0fa-4e9f-b80f-a3418f8883a4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:23.184100 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-dd0680a5-f0fa-4e9f-b80f-a3418f8883a4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:23.186121 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-dd0680a5-f0fa-4e9f-b80f-a3418f8883a4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:23.187391 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-dd0680a5-f0fa-4e9f-b80f-a3418f8883a4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:23.189300 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-dd0680a5-f0fa-4e9f-b80f-a3418f8883a4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:23.407469 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-dd0680a5-f0fa-4e9f-b80f-a3418f8883a4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:23.425053 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-dd0680a5-f0fa-4e9f-b80f-a3418f8883a4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.264322 Oct 28 09:46:23.431699 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 131/245] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:23 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:23.797333 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3e271484-e55f-42b4-be00-6372f5afd9f5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:23.797945 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3e271484-e55f-42b4-be00-6372f5afd9f5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:23.820811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3e271484-e55f-42b4-be00-6372f5afd9f5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:23.825107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3e271484-e55f-42b4-be00-6372f5afd9f5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:23.825544 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3e271484-e55f-42b4-be00-6372f5afd9f5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:23.826276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3e271484-e55f-42b4-be00-6372f5afd9f5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:23.826746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3e271484-e55f-42b4-be00-6372f5afd9f5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:23.827169 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3e271484-e55f-42b4-be00-6372f5afd9f5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:23.856641 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-3e271484-e55f-42b4-be00-6372f5afd9f5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] HTTP exception thrown: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3e271484-e55f-42b4-be00-6372f5afd9f5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Request method failure captured: Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc HTTP/1.1 Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6SqFJdV13WJaB9Vi5iVA-J12ScGjTA4tSa5jiTwY2XbSMyH0efH5SHhum5wso2BEDj7ADpjROjjNnYOLKTXIhSqKb9YOp9BQ6NQlWcIHl2Nzzaan2CIPukDQPPZlXQag7QxqGECV5rCLqAXDojv4ygzMltdvMuy53Jehblt_1wjZA0 Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 41516f10d050497e82056ddd1be5981c Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-DeleteServersTestJSON-676189331 Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:46:23.867985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:46:23.869590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/41516f10d050497e82056ddd1be5981c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/41516f10d050497e82056ddd1be5981c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_41516f10d050497e82056ddd1be5981c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/41516f10d050497e82056ddd1be5981c"}]}] Oct 28 09:46:23.869590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-DeleteServersTestJSON-676189331 Oct 28 09:46:23.869590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 41516f10d050497e82056ddd1be5981c Oct 28 09:46:23.869590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-DeleteServersTestJSON-676189331 Oct 28 09:46:23.869590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-DeleteServersTestJSON-676189331-project Oct 28 09:46:23.869590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:46:23.869590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:46:23.869590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e451d264e4964409b09e0b96ae8b983e Oct 28 09:46:23.869590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-DeleteServersTestJSON-676189331-project Oct 28 09:46:23.869590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:46:23.869590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:23.869590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '088d0e80-3838-4ada-938b-f26f194d5abc'} Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:46:23.870448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:23.871777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:46:23.872934 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3e271484-e55f-42b4-be00-6372f5afd9f5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Returning 404 to user: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:46:23.872934 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3e271484-e55f-42b4-be00-6372f5afd9f5 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 404 len: 111 microversion: 2.1 time: 0.076123 Oct 28 09:46:23.895069 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 115/246] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:23 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 111 bytes in 95 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:46:24.082041 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-0d3ab712-e3c1-4c98-967b-52ef968e9ef7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:24.087680 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-0d3ab712-e3c1-4c98-967b-52ef968e9ef7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:24.103720 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d3ab712-e3c1-4c98-967b-52ef968e9ef7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:24.104514 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d3ab712-e3c1-4c98-967b-52ef968e9ef7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:24.105147 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d3ab712-e3c1-4c98-967b-52ef968e9ef7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:24.106489 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d3ab712-e3c1-4c98-967b-52ef968e9ef7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:24.107897 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d3ab712-e3c1-4c98-967b-52ef968e9ef7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:24.108382 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d3ab712-e3c1-4c98-967b-52ef968e9ef7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:24.138607 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.wsgi [None req-0d3ab712-e3c1-4c98-967b-52ef968e9ef7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] HTTP exception thrown: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-0d3ab712-e3c1-4c98-967b-52ef968e9ef7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Request method failure captured: Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: request: GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc/os-volume_attachments HTTP/1.1 Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept: application/json Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept-Encoding: identity Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Connection: close Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Length: 0 Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Type: application/json Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Host: 10.222.0.25 Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Openstack-System-Scope: None Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: User-Agent: python-urllib3/1.26.12 Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Auth-Token: gAAAAABjW6SqFJdV13WJaB9Vi5iVA-J12ScGjTA4tSa5jiTwY2XbSMyH0efH5SHhum5wso2BEDj7ADpjROjjNnYOLKTXIhSqKb9YOp9BQ6NQlWcIHl2Nzzaan2CIPukDQPPZlXQag7QxqGECV5rCLqAXDojv4ygzMltdvMuy53Jehblt_1wjZA0 Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Id: None Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Name: None Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Identity-Status: Confirmed Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Is-Admin-Project: True Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Id: default Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Name: Default Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Id: 41516f10d050497e82056ddd1be5981c Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Name: tempest-DeleteServersTestJSON-676189331 Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Role: member,reader Oct 28 09:46:24.144972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Roles: member,reader Oct 28 09:46:24.146444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/41516f10d050497e82056ddd1be5981c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/41516f10d050497e82056ddd1be5981c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_41516f10d050497e82056ddd1be5981c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/41516f10d050497e82056ddd1be5981c"}]}] Oct 28 09:46:24.146444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant: tempest-DeleteServersTestJSON-676189331 Oct 28 09:46:24.146444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Id: 41516f10d050497e82056ddd1be5981c Oct 28 09:46:24.146444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Name: tempest-DeleteServersTestJSON-676189331 Oct 28 09:46:24.146444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User: tempest-DeleteServersTestJSON-676189331-project Oct 28 09:46:24.146444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Id: default Oct 28 09:46:24.146444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Name: Default Oct 28 09:46:24.146444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Id: e451d264e4964409b09e0b96ae8b983e Oct 28 09:46:24.146444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Name: tempest-DeleteServersTestJSON-676189331-project Oct 28 09:46:24.146444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: method: > Oct 28 09:46:24.146444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: exception: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:24.146444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: action_args: {'server_id': '088d0e80-3838-4ada-938b-f26f194d5abc'} Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:46:24.147129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:24.149772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:46:24.153025 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-0d3ab712-e3c1-4c98-967b-52ef968e9ef7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Returning 404 to user: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. {{(pid=127029) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:46:24.153025 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-0d3ab712-e3c1-4c98-967b-52ef968e9ef7 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.073588 Oct 28 09:46:24.153025 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 132/247] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:46:24 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc/os-volume_attachments => generated 111 bytes in 75 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:46:24.454632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-70b2c529-0cb0-459f-b1d2-58ac45b695fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:24.455526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-70b2c529-0cb0-459f-b1d2-58ac45b695fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:24.465161 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-70b2c529-0cb0-459f-b1d2-58ac45b695fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:24.465645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-70b2c529-0cb0-459f-b1d2-58ac45b695fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:24.466078 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-70b2c529-0cb0-459f-b1d2-58ac45b695fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:24.466843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-70b2c529-0cb0-459f-b1d2-58ac45b695fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:24.467395 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-70b2c529-0cb0-459f-b1d2-58ac45b695fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:24.467828 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-70b2c529-0cb0-459f-b1d2-58ac45b695fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:24.876974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-70b2c529-0cb0-459f-b1d2-58ac45b695fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:24.885885 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-70b2c529-0cb0-459f-b1d2-58ac45b695fd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.435499 Oct 28 09:46:24.889074 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 116/248] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:24 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:25.542988 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2568a32d-2e9a-490e-8b88-c9349e2b98c1 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:25.544027 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2568a32d-2e9a-490e-8b88-c9349e2b98c1 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:25.556866 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2568a32d-2e9a-490e-8b88-c9349e2b98c1 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:25.557617 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2568a32d-2e9a-490e-8b88-c9349e2b98c1 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:25.558201 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2568a32d-2e9a-490e-8b88-c9349e2b98c1 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:25.559084 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2568a32d-2e9a-490e-8b88-c9349e2b98c1 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:25.559677 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2568a32d-2e9a-490e-8b88-c9349e2b98c1 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:25.560441 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2568a32d-2e9a-490e-8b88-c9349e2b98c1 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:25.586829 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.wsgi [None req-2568a32d-2e9a-490e-8b88-c9349e2b98c1 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] HTTP exception thrown: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2568a32d-2e9a-490e-8b88-c9349e2b98c1 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Request method failure captured: Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: request: DELETE /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc HTTP/1.1 Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept: application/json Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept-Encoding: identity Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Connection: close Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Length: 0 Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Type: application/json Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Host: 10.222.0.25 Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Openstack-System-Scope: None Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: User-Agent: python-urllib3/1.26.12 Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Auth-Token: gAAAAABjW6SqFJdV13WJaB9Vi5iVA-J12ScGjTA4tSa5jiTwY2XbSMyH0efH5SHhum5wso2BEDj7ADpjROjjNnYOLKTXIhSqKb9YOp9BQ6NQlWcIHl2Nzzaan2CIPukDQPPZlXQag7QxqGECV5rCLqAXDojv4ygzMltdvMuy53Jehblt_1wjZA0 Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Id: None Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Name: None Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Identity-Status: Confirmed Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Is-Admin-Project: True Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Id: default Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Name: Default Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Id: 41516f10d050497e82056ddd1be5981c Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Name: tempest-DeleteServersTestJSON-676189331 Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Role: member,reader Oct 28 09:46:25.589003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Roles: member,reader Oct 28 09:46:25.590384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/41516f10d050497e82056ddd1be5981c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/41516f10d050497e82056ddd1be5981c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_41516f10d050497e82056ddd1be5981c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/41516f10d050497e82056ddd1be5981c"}]}] Oct 28 09:46:25.590384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant: tempest-DeleteServersTestJSON-676189331 Oct 28 09:46:25.590384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Id: 41516f10d050497e82056ddd1be5981c Oct 28 09:46:25.590384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Name: tempest-DeleteServersTestJSON-676189331 Oct 28 09:46:25.590384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User: tempest-DeleteServersTestJSON-676189331-project Oct 28 09:46:25.590384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Id: default Oct 28 09:46:25.590384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Name: Default Oct 28 09:46:25.590384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Id: e451d264e4964409b09e0b96ae8b983e Oct 28 09:46:25.590384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Name: tempest-DeleteServersTestJSON-676189331-project Oct 28 09:46:25.590384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: method: > Oct 28 09:46:25.590384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: exception: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:25.590384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: action_args: {'id': '088d0e80-3838-4ada-938b-f26f194d5abc'} Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:46:25.591296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:25.592642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:46:25.594777 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2568a32d-2e9a-490e-8b88-c9349e2b98c1 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Returning 404 to user: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. {{(pid=127029) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:46:25.595893 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2568a32d-2e9a-490e-8b88-c9349e2b98c1 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "DELETE /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 404 len: 111 microversion: 2.1 time: 0.057134 Oct 28 09:46:25.596451 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 133/249] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:46:25 2022] DELETE /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:46:25.604647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-356ad695-95a3-48c3-9c9e-c91a549a3301 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:25.605657 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-356ad695-95a3-48c3-9c9e-c91a549a3301 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] [instance: 088d0e80-3838-4ada-938b-f26f194d5abc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:25.611819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-356ad695-95a3-48c3-9c9e-c91a549a3301 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:25.612296 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-356ad695-95a3-48c3-9c9e-c91a549a3301 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:25.612722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-356ad695-95a3-48c3-9c9e-c91a549a3301 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:25.616798 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-356ad695-95a3-48c3-9c9e-c91a549a3301 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:25.617268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-356ad695-95a3-48c3-9c9e-c91a549a3301 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:25.617696 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-356ad695-95a3-48c3-9c9e-c91a549a3301 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:25.643892 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-356ad695-95a3-48c3-9c9e-c91a549a3301 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] HTTP exception thrown: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-356ad695-95a3-48c3-9c9e-c91a549a3301 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Request method failure captured: Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc HTTP/1.1 Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6SqFJdV13WJaB9Vi5iVA-J12ScGjTA4tSa5jiTwY2XbSMyH0efH5SHhum5wso2BEDj7ADpjROjjNnYOLKTXIhSqKb9YOp9BQ6NQlWcIHl2Nzzaan2CIPukDQPPZlXQag7QxqGECV5rCLqAXDojv4ygzMltdvMuy53Jehblt_1wjZA0 Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 41516f10d050497e82056ddd1be5981c Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-DeleteServersTestJSON-676189331 Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:46:25.648953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:46:25.650433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/41516f10d050497e82056ddd1be5981c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/41516f10d050497e82056ddd1be5981c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_41516f10d050497e82056ddd1be5981c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/41516f10d050497e82056ddd1be5981c"}]}] Oct 28 09:46:25.650433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-DeleteServersTestJSON-676189331 Oct 28 09:46:25.650433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 41516f10d050497e82056ddd1be5981c Oct 28 09:46:25.650433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-DeleteServersTestJSON-676189331 Oct 28 09:46:25.650433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-DeleteServersTestJSON-676189331-project Oct 28 09:46:25.650433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:46:25.650433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:46:25.650433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e451d264e4964409b09e0b96ae8b983e Oct 28 09:46:25.650433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-DeleteServersTestJSON-676189331-project Oct 28 09:46:25.650433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:46:25.650433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:25.650433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '088d0e80-3838-4ada-938b-f26f194d5abc'} Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:46:25.651231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. Oct 28 09:46:25.652509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:46:25.654726 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-356ad695-95a3-48c3-9c9e-c91a549a3301 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] Returning 404 to user: Instance 088d0e80-3838-4ada-938b-f26f194d5abc could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:46:25.656788 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-356ad695-95a3-48c3-9c9e-c91a549a3301 tempest-DeleteServersTestJSON-676189331 tempest-DeleteServersTestJSON-676189331-project] 10.222.0.25 "GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc" status: 404 len: 111 microversion: 2.1 time: 0.055914 Oct 28 09:46:25.657356 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 117/250] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:25 2022] GET /compute/v2.1/servers/088d0e80-3838-4ada-938b-f26f194d5abc => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:46:25.920788 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-98b033b7-49e1-4ae6-964a-3338231dc90f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:25.923013 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-98b033b7-49e1-4ae6-964a-3338231dc90f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:25.962528 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-98b033b7-49e1-4ae6-964a-3338231dc90f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:25.963378 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-98b033b7-49e1-4ae6-964a-3338231dc90f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:25.964060 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-98b033b7-49e1-4ae6-964a-3338231dc90f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:25.969057 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-98b033b7-49e1-4ae6-964a-3338231dc90f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:25.974460 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-98b033b7-49e1-4ae6-964a-3338231dc90f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:25.975168 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-98b033b7-49e1-4ae6-964a-3338231dc90f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:26.328498 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-98b033b7-49e1-4ae6-964a-3338231dc90f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:26.350810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-98b033b7-49e1-4ae6-964a-3338231dc90f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.440277 Oct 28 09:46:26.351969 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 134/251] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:25 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:27.381483 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f821ff4a-df20-4f37-8a8c-ce0fc46ccd2a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:27.382824 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f821ff4a-df20-4f37-8a8c-ce0fc46ccd2a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:27.391606 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f821ff4a-df20-4f37-8a8c-ce0fc46ccd2a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:27.392474 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f821ff4a-df20-4f37-8a8c-ce0fc46ccd2a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:27.393644 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f821ff4a-df20-4f37-8a8c-ce0fc46ccd2a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:27.394902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f821ff4a-df20-4f37-8a8c-ce0fc46ccd2a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:27.395575 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f821ff4a-df20-4f37-8a8c-ce0fc46ccd2a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:27.396568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f821ff4a-df20-4f37-8a8c-ce0fc46ccd2a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:27.586816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f821ff4a-df20-4f37-8a8c-ce0fc46ccd2a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:27.597354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f821ff4a-df20-4f37-8a8c-ce0fc46ccd2a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.219635 Oct 28 09:46:27.610647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 118/252] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:27 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:28.620807 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-a8c3d13b-1784-4380-ac58-5ad78ef361ec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:28.621525 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-a8c3d13b-1784-4380-ac58-5ad78ef361ec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:28.629390 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a8c3d13b-1784-4380-ac58-5ad78ef361ec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:28.630167 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a8c3d13b-1784-4380-ac58-5ad78ef361ec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:28.633369 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a8c3d13b-1784-4380-ac58-5ad78ef361ec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:28.635341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a8c3d13b-1784-4380-ac58-5ad78ef361ec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:28.638068 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a8c3d13b-1784-4380-ac58-5ad78ef361ec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:28.641714 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a8c3d13b-1784-4380-ac58-5ad78ef361ec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:28.956557 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-a8c3d13b-1784-4380-ac58-5ad78ef361ec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:28.972524 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-a8c3d13b-1784-4380-ac58-5ad78ef361ec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.353300 Oct 28 09:46:28.973458 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 135/253] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:28 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:30.002230 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-36b15d26-0223-4108-81d8-b9500b2a09f2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:30.003287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-36b15d26-0223-4108-81d8-b9500b2a09f2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:30.032314 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36b15d26-0223-4108-81d8-b9500b2a09f2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:30.033344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36b15d26-0223-4108-81d8-b9500b2a09f2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:30.034296 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36b15d26-0223-4108-81d8-b9500b2a09f2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:30.035677 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36b15d26-0223-4108-81d8-b9500b2a09f2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:30.037667 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36b15d26-0223-4108-81d8-b9500b2a09f2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:30.039645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36b15d26-0223-4108-81d8-b9500b2a09f2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:30.507863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-36b15d26-0223-4108-81d8-b9500b2a09f2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:30.530956 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-36b15d26-0223-4108-81d8-b9500b2a09f2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.535811 Oct 28 09:46:30.548117 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 119/254] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:29 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:31.560064 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-69285d0a-4e5a-45d6-9af3-b13f0cbb29da tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:31.561415 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-69285d0a-4e5a-45d6-9af3-b13f0cbb29da tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:31.571471 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-69285d0a-4e5a-45d6-9af3-b13f0cbb29da tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:31.572331 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-69285d0a-4e5a-45d6-9af3-b13f0cbb29da tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:31.573076 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-69285d0a-4e5a-45d6-9af3-b13f0cbb29da tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:31.573958 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-69285d0a-4e5a-45d6-9af3-b13f0cbb29da tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:31.578711 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-69285d0a-4e5a-45d6-9af3-b13f0cbb29da tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:31.583743 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-69285d0a-4e5a-45d6-9af3-b13f0cbb29da tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:32.390623 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-69285d0a-4e5a-45d6-9af3-b13f0cbb29da tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:32.420202 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-69285d0a-4e5a-45d6-9af3-b13f0cbb29da tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.864164 Oct 28 09:46:32.423571 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 136/255] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:31 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 868 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:33.449578 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1ce2b193-31e8-4fb0-8866-378d0028db38 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:33.450491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1ce2b193-31e8-4fb0-8866-378d0028db38 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:33.470477 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ce2b193-31e8-4fb0-8866-378d0028db38 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:33.470477 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ce2b193-31e8-4fb0-8866-378d0028db38 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:33.471359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ce2b193-31e8-4fb0-8866-378d0028db38 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:33.473182 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ce2b193-31e8-4fb0-8866-378d0028db38 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:33.474481 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ce2b193-31e8-4fb0-8866-378d0028db38 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:33.475144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ce2b193-31e8-4fb0-8866-378d0028db38 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:33.655778 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1ce2b193-31e8-4fb0-8866-378d0028db38 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:33.665177 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1ce2b193-31e8-4fb0-8866-378d0028db38 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.223745 Oct 28 09:46:33.678419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 120/256] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:33 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:34.689730 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-256e2cf0-707c-4619-9a73-3c7f514fda3f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:34.690387 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-256e2cf0-707c-4619-9a73-3c7f514fda3f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:34.697905 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-256e2cf0-707c-4619-9a73-3c7f514fda3f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:34.698308 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-256e2cf0-707c-4619-9a73-3c7f514fda3f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:34.698782 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-256e2cf0-707c-4619-9a73-3c7f514fda3f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:34.699491 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-256e2cf0-707c-4619-9a73-3c7f514fda3f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:34.700150 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-256e2cf0-707c-4619-9a73-3c7f514fda3f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:34.700564 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-256e2cf0-707c-4619-9a73-3c7f514fda3f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:34.884789 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-256e2cf0-707c-4619-9a73-3c7f514fda3f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:34.892784 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-256e2cf0-707c-4619-9a73-3c7f514fda3f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.207319 Oct 28 09:46:34.912280 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 137/257] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:34 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:35.925434 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-92aae117-f3c6-457b-98ba-c57c4725a24d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:35.926072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-92aae117-f3c6-457b-98ba-c57c4725a24d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:35.932809 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-92aae117-f3c6-457b-98ba-c57c4725a24d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:35.933557 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-92aae117-f3c6-457b-98ba-c57c4725a24d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:35.934185 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-92aae117-f3c6-457b-98ba-c57c4725a24d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:35.935123 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-92aae117-f3c6-457b-98ba-c57c4725a24d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:35.935798 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-92aae117-f3c6-457b-98ba-c57c4725a24d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:35.936412 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-92aae117-f3c6-457b-98ba-c57c4725a24d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:36.081858 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-92aae117-f3c6-457b-98ba-c57c4725a24d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:36.090231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-92aae117-f3c6-457b-98ba-c57c4725a24d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.176075 Oct 28 09:46:36.091082 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 121/258] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:35 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:37.116624 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-9cac9393-0f5c-4d1d-b24f-30d4870fdab6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:37.117967 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-9cac9393-0f5c-4d1d-b24f-30d4870fdab6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:37.126939 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9cac9393-0f5c-4d1d-b24f-30d4870fdab6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:37.127667 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9cac9393-0f5c-4d1d-b24f-30d4870fdab6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:37.128349 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9cac9393-0f5c-4d1d-b24f-30d4870fdab6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:37.129341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9cac9393-0f5c-4d1d-b24f-30d4870fdab6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:37.130592 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9cac9393-0f5c-4d1d-b24f-30d4870fdab6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:37.133525 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9cac9393-0f5c-4d1d-b24f-30d4870fdab6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:37.322535 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-9cac9393-0f5c-4d1d-b24f-30d4870fdab6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:37.334267 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-9cac9393-0f5c-4d1d-b24f-30d4870fdab6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.222301 Oct 28 09:46:37.355375 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 138/259] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:37 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:38.364659 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-eeb5c3a1-fc09-49bc-8ac2-f810b506dfcc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:38.366118 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-eeb5c3a1-fc09-49bc-8ac2-f810b506dfcc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:38.375615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eeb5c3a1-fc09-49bc-8ac2-f810b506dfcc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:38.375897 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eeb5c3a1-fc09-49bc-8ac2-f810b506dfcc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:38.376342 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eeb5c3a1-fc09-49bc-8ac2-f810b506dfcc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:38.377070 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eeb5c3a1-fc09-49bc-8ac2-f810b506dfcc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:38.377529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eeb5c3a1-fc09-49bc-8ac2-f810b506dfcc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:38.377951 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eeb5c3a1-fc09-49bc-8ac2-f810b506dfcc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:38.534422 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-eeb5c3a1-fc09-49bc-8ac2-f810b506dfcc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:38.543823 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-eeb5c3a1-fc09-49bc-8ac2-f810b506dfcc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.187699 Oct 28 09:46:38.545019 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 122/260] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:38 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:39.582994 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-86a1a937-7053-4eab-99a8-ffd35e290f3a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:39.584459 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-86a1a937-7053-4eab-99a8-ffd35e290f3a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:39.597412 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-86a1a937-7053-4eab-99a8-ffd35e290f3a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:39.604358 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-86a1a937-7053-4eab-99a8-ffd35e290f3a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:39.605150 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-86a1a937-7053-4eab-99a8-ffd35e290f3a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:39.605946 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-86a1a937-7053-4eab-99a8-ffd35e290f3a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:39.606722 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-86a1a937-7053-4eab-99a8-ffd35e290f3a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:39.607239 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-86a1a937-7053-4eab-99a8-ffd35e290f3a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:39.776908 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-86a1a937-7053-4eab-99a8-ffd35e290f3a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:39.788874 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-86a1a937-7053-4eab-99a8-ffd35e290f3a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.219974 Oct 28 09:46:39.791023 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 139/261] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:39 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:40.059510 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f45dccfb-eaea-4e60-8e08-e5b873648e76 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Action: 'create', calling method: .version_select at 0x7f271a5559d0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-498428939", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC+H72ep7oHbzgtXy4exlsHLFlmpYZff9lDsdI+fpAsXEM1/FNFFWvuejIAV9esyS9HvxjWgSFAxAENYtNDGet7SRcU8sMjyXLsY3k5q4VKj5EbGEow6jmHIJEeuSvRWTA=="}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:46:40.077088 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-f45dccfb-eaea-4e60-8e08-e5b873648e76 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Getting quotas for project 6f99f17a6cab46039419eb3b44bdd142. Resources: {'key_pairs'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:46:40.081233 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-f45dccfb-eaea-4e60-8e08-e5b873648e76 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Getting quotas for user 23acf07cede847a69ca839ee200accd3 and project 6f99f17a6cab46039419eb3b44bdd142. Resources: {'key_pairs'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:46:40.121260 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f45dccfb-eaea-4e60-8e08-e5b873648e76 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.065872 Oct 28 09:46:40.121855 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 123/262] 10.222.0.25 () {60 vars in 1159 bytes} [Fri Oct 28 09:46:40 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:46:40.237741 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1998121339", "imageRef": "578dd95e-64e4-493e-af79-a85e75b060d3", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-498428939", "networks": [{"uuid": "4312b894-27db-4853-89d7-6a6630738a6b"}]}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:46:40.293927 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.network.neutron [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] validate_networks() for [('4312b894-27db-4853-89d7-6a6630738a6b', None, None, None, None, None)] {{(pid=127029) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:46:40.718780 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Flavor limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:46:40.719354 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Image limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:46:40.719645 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Flavor pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:46:40.720044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Image pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:46:40.720444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:46:40.733808 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:40.734135 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:40.734661 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:40.735453 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:40.735967 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:40.736641 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:40.761958 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Getting quotas for project 6f99f17a6cab46039419eb3b44bdd142. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:46:40.766513 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Getting quotas for user 23acf07cede847a69ca839ee200accd3 and project 6f99f17a6cab46039419eb3b44bdd142. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:46:40.779104 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Going to run 1 instances... {{(pid=127029) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:46:40.781360 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='578dd95e-64e4-493e-af79-a85e75b060d3',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127029) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:46:40.813791 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-47927020-92e4-47ed-a8e9-3870285f4177 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:40.814650 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-47927020-92e4-47ed-a8e9-3870285f4177 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:40.823811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-47927020-92e4-47ed-a8e9-3870285f4177 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:40.824573 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-47927020-92e4-47ed-a8e9-3870285f4177 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:40.825166 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-47927020-92e4-47ed-a8e9-3870285f4177 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:40.826127 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-47927020-92e4-47ed-a8e9-3870285f4177 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:40.826613 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-47927020-92e4-47ed-a8e9-3870285f4177 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:40.827073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-47927020-92e4-47ed-a8e9-3870285f4177 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:40.841023 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6465c35a-8004-4737-9cb3-723bbff84fed tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.607352 Oct 28 09:46:40.842072 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 140/263] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:46:40 2022] POST /compute/v2.1/servers => generated 380 bytes in 609 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 09:46:40.855516 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2cf4821e-109e-4f58-927f-3e5d4ef7ff9d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:40.856471 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2cf4821e-109e-4f58-927f-3e5d4ef7ff9d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:40.938472 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2cf4821e-109e-4f58-927f-3e5d4ef7ff9d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:40.948074 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2cf4821e-109e-4f58-927f-3e5d4ef7ff9d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1258 microversion: 2.1 time: 0.097108 Oct 28 09:46:40.948649 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 141/264] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:40 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1258 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:41.009494 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-47927020-92e4-47ed-a8e9-3870285f4177 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:41.018258 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-47927020-92e4-47ed-a8e9-3870285f4177 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.208895 Oct 28 09:46:41.018981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 124/265] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:40 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:41.972971 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c135e78c-7ee1-48bf-a644-8f85b4195b4f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:41.973675 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c135e78c-7ee1-48bf-a644-8f85b4195b4f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:41.980270 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c135e78c-7ee1-48bf-a644-8f85b4195b4f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:41.980749 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c135e78c-7ee1-48bf-a644-8f85b4195b4f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:41.981178 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c135e78c-7ee1-48bf-a644-8f85b4195b4f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:41.981868 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c135e78c-7ee1-48bf-a644-8f85b4195b4f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:41.982311 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c135e78c-7ee1-48bf-a644-8f85b4195b4f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:41.982758 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c135e78c-7ee1-48bf-a644-8f85b4195b4f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:42.043699 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7b0003b9-1b53-42e1-80d5-50d0da6b8f3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:42.044316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7b0003b9-1b53-42e1-80d5-50d0da6b8f3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:42.057897 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c135e78c-7ee1-48bf-a644-8f85b4195b4f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:42.058213 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7b0003b9-1b53-42e1-80d5-50d0da6b8f3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:42.058795 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7b0003b9-1b53-42e1-80d5-50d0da6b8f3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:42.059738 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7b0003b9-1b53-42e1-80d5-50d0da6b8f3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:42.060886 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7b0003b9-1b53-42e1-80d5-50d0da6b8f3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:42.061314 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:42.062360 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7b0003b9-1b53-42e1-80d5-50d0da6b8f3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:42.062936 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7b0003b9-1b53-42e1-80d5-50d0da6b8f3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:42.071223 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c135e78c-7ee1-48bf-a644-8f85b4195b4f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1310 microversion: 2.1 time: 0.101813 Oct 28 09:46:42.072824 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 142/266] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:41 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1310 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:42.203086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7b0003b9-1b53-42e1-80d5-50d0da6b8f3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:42.212240 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7b0003b9-1b53-42e1-80d5-50d0da6b8f3d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.174893 Oct 28 09:46:42.212818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 125/267] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:42 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:43.095375 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-ed9dbbee-d1f4-42c3-ac23-edb8bb7da153 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:43.096001 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-ed9dbbee-d1f4-42c3-ac23-edb8bb7da153 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:43.101864 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ed9dbbee-d1f4-42c3-ac23-edb8bb7da153 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:43.102338 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ed9dbbee-d1f4-42c3-ac23-edb8bb7da153 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:43.102810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ed9dbbee-d1f4-42c3-ac23-edb8bb7da153 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:43.103488 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ed9dbbee-d1f4-42c3-ac23-edb8bb7da153 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:43.103948 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ed9dbbee-d1f4-42c3-ac23-edb8bb7da153 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:43.104343 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ed9dbbee-d1f4-42c3-ac23-edb8bb7da153 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:43.167895 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-ed9dbbee-d1f4-42c3-ac23-edb8bb7da153 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:43.176653 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-ed9dbbee-d1f4-42c3-ac23-edb8bb7da153 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1310 microversion: 2.1 time: 0.085395 Oct 28 09:46:43.177225 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 143/268] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:43 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1310 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:43.235149 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c9af9772-4399-478a-bacf-fab7bc1cee53 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:43.236809 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c9af9772-4399-478a-bacf-fab7bc1cee53 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:43.242566 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9af9772-4399-478a-bacf-fab7bc1cee53 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:43.243055 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9af9772-4399-478a-bacf-fab7bc1cee53 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:43.243493 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9af9772-4399-478a-bacf-fab7bc1cee53 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:43.244173 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9af9772-4399-478a-bacf-fab7bc1cee53 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:43.244614 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9af9772-4399-478a-bacf-fab7bc1cee53 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:43.245024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9af9772-4399-478a-bacf-fab7bc1cee53 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:43.385305 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c9af9772-4399-478a-bacf-fab7bc1cee53 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:43.393051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c9af9772-4399-478a-bacf-fab7bc1cee53 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1522 microversion: 2.1 time: 0.161750 Oct 28 09:46:43.393618 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 126/269] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:43 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:44.201696 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-4951f197-8e92-4a9a-8e01-587456feb5aa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:44.203361 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-4951f197-8e92-4a9a-8e01-587456feb5aa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:44.211682 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4951f197-8e92-4a9a-8e01-587456feb5aa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:44.212520 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4951f197-8e92-4a9a-8e01-587456feb5aa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:44.212867 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4951f197-8e92-4a9a-8e01-587456feb5aa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:44.213494 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4951f197-8e92-4a9a-8e01-587456feb5aa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:44.213940 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4951f197-8e92-4a9a-8e01-587456feb5aa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:44.214362 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4951f197-8e92-4a9a-8e01-587456feb5aa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:44.286341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-4951f197-8e92-4a9a-8e01-587456feb5aa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:44.294199 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-4951f197-8e92-4a9a-8e01-587456feb5aa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1316 microversion: 2.1 time: 0.096819 Oct 28 09:46:44.294796 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 144/270] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:44 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1316 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:44.415940 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-592a87bb-4ffd-407b-8a6d-2c60946fe4be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:44.416534 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-592a87bb-4ffd-407b-8a6d-2c60946fe4be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:44.423444 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-592a87bb-4ffd-407b-8a6d-2c60946fe4be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:44.423922 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-592a87bb-4ffd-407b-8a6d-2c60946fe4be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:44.424345 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-592a87bb-4ffd-407b-8a6d-2c60946fe4be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:44.425334 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-592a87bb-4ffd-407b-8a6d-2c60946fe4be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:44.425770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-592a87bb-4ffd-407b-8a6d-2c60946fe4be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:44.426777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-592a87bb-4ffd-407b-8a6d-2c60946fe4be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:44.635289 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-592a87bb-4ffd-407b-8a6d-2c60946fe4be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:44.644571 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-592a87bb-4ffd-407b-8a6d-2c60946fe4be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1502 microversion: 2.1 time: 0.232555 Oct 28 09:46:44.645981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 127/271] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:44 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1502 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:44.671886 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-063563ab-1554-4a3b-9f1a-0dcf760481e8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:44.672645 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-063563ab-1554-4a3b-9f1a-0dcf760481e8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:44.679056 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-063563ab-1554-4a3b-9f1a-0dcf760481e8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:44.679449 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-063563ab-1554-4a3b-9f1a-0dcf760481e8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:44.680083 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-063563ab-1554-4a3b-9f1a-0dcf760481e8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:44.681505 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-063563ab-1554-4a3b-9f1a-0dcf760481e8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:44.682139 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-063563ab-1554-4a3b-9f1a-0dcf760481e8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:44.682758 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-063563ab-1554-4a3b-9f1a-0dcf760481e8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:44.936850 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-063563ab-1554-4a3b-9f1a-0dcf760481e8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:44.946053 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-063563ab-1554-4a3b-9f1a-0dcf760481e8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1517 microversion: 2.1 time: 0.279115 Oct 28 09:46:44.946685 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 145/272] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:44 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1517 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:45.323254 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7af09316-46aa-4ef5-a9a8-0c48b5ed9615 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:45.323882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7af09316-46aa-4ef5-a9a8-0c48b5ed9615 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:45.347377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7af09316-46aa-4ef5-a9a8-0c48b5ed9615 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:45.349112 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7af09316-46aa-4ef5-a9a8-0c48b5ed9615 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:45.350866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7af09316-46aa-4ef5-a9a8-0c48b5ed9615 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:45.352109 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7af09316-46aa-4ef5-a9a8-0c48b5ed9615 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:45.352601 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7af09316-46aa-4ef5-a9a8-0c48b5ed9615 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:45.354030 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7af09316-46aa-4ef5-a9a8-0c48b5ed9615 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:45.638344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7af09316-46aa-4ef5-a9a8-0c48b5ed9615 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:45.648575 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7af09316-46aa-4ef5-a9a8-0c48b5ed9615 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.331797 Oct 28 09:46:45.649795 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 128/273] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:45 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:46.661758 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b226a398-4a6d-4644-a6f1-0e0c44695729 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:46:46.667672 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b226a398-4a6d-4644-a6f1-0e0c44695729 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:46.675272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-97d629e5-56c0-4266-b5d0-e70666847720 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:46.675787 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-97d629e5-56c0-4266-b5d0-e70666847720 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:46.683584 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b226a398-4a6d-4644-a6f1-0e0c44695729 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:46.683947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-97d629e5-56c0-4266-b5d0-e70666847720 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:46.684126 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b226a398-4a6d-4644-a6f1-0e0c44695729 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:46.684429 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-97d629e5-56c0-4266-b5d0-e70666847720 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:46.684566 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b226a398-4a6d-4644-a6f1-0e0c44695729 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:46.684855 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-97d629e5-56c0-4266-b5d0-e70666847720 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:46.685272 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b226a398-4a6d-4644-a6f1-0e0c44695729 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:46.685534 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-97d629e5-56c0-4266-b5d0-e70666847720 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:46.685720 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b226a398-4a6d-4644-a6f1-0e0c44695729 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:46.685974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-97d629e5-56c0-4266-b5d0-e70666847720 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:46.686172 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b226a398-4a6d-4644-a6f1-0e0c44695729 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:46.686426 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-97d629e5-56c0-4266-b5d0-e70666847720 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:46.716381 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.objects.instance [None req-b226a398-4a6d-4644-a6f1-0e0c44695729 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lazy-loading 'flavor' on Instance uuid 3b7d7553-45b6-4a10-868e-b653d211245d {{(pid=127029) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:46:46.883629 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-97d629e5-56c0-4266-b5d0-e70666847720 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:46.902159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-97d629e5-56c0-4266-b5d0-e70666847720 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.232146 Oct 28 09:46:46.902574 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b226a398-4a6d-4644-a6f1-0e0c44695729 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "POST /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d/action" status: 200 len: 32 microversion: 2.1 time: 0.247130 Oct 28 09:46:46.914433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 129/274] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:46 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:46.914638 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 146/275] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:46:46 2022] POST /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d/action => generated 32 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 09:46:46.924891 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e3e0f122-0699-47cc-9971-893f430b4741 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:46.925594 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e3e0f122-0699-47cc-9971-893f430b4741 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:46.940165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e3e0f122-0699-47cc-9971-893f430b4741 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:46.940856 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e3e0f122-0699-47cc-9971-893f430b4741 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:46.941192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e3e0f122-0699-47cc-9971-893f430b4741 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:46.941948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e3e0f122-0699-47cc-9971-893f430b4741 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:46.942427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e3e0f122-0699-47cc-9971-893f430b4741 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:46.942872 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e3e0f122-0699-47cc-9971-893f430b4741 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:47.141420 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e3e0f122-0699-47cc-9971-893f430b4741 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:47.150067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e3e0f122-0699-47cc-9971-893f430b4741 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.241111 Oct 28 09:46:47.164086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 130/276] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:46 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:47.934225 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-40be05c4-89b9-45d4-bc01-4e1bad6849e8 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:47.935967 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-40be05c4-89b9-45d4-bc01-4e1bad6849e8 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:47.944480 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40be05c4-89b9-45d4-bc01-4e1bad6849e8 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:47.945458 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40be05c4-89b9-45d4-bc01-4e1bad6849e8 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:47.946925 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40be05c4-89b9-45d4-bc01-4e1bad6849e8 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:47.947953 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40be05c4-89b9-45d4-bc01-4e1bad6849e8 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:47.948454 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40be05c4-89b9-45d4-bc01-4e1bad6849e8 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:47.948914 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-40be05c4-89b9-45d4-bc01-4e1bad6849e8 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:48.173259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e52e0476-a8b3-47a3-89cd-0bbb6d2109cd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:48.173878 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e52e0476-a8b3-47a3-89cd-0bbb6d2109cd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:48.182116 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e52e0476-a8b3-47a3-89cd-0bbb6d2109cd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:48.182985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e52e0476-a8b3-47a3-89cd-0bbb6d2109cd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:48.183883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e52e0476-a8b3-47a3-89cd-0bbb6d2109cd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:48.184909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e52e0476-a8b3-47a3-89cd-0bbb6d2109cd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:48.185512 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e52e0476-a8b3-47a3-89cd-0bbb6d2109cd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:48.186356 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e52e0476-a8b3-47a3-89cd-0bbb6d2109cd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:48.428726 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-40be05c4-89b9-45d4-bc01-4e1bad6849e8 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:48.440480 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-40be05c4-89b9-45d4-bc01-4e1bad6849e8 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.514295 Oct 28 09:46:48.443293 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 147/277] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:47 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:48.458926 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e52e0476-a8b3-47a3-89cd-0bbb6d2109cd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:48.471878 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e52e0476-a8b3-47a3-89cd-0bbb6d2109cd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.301866 Oct 28 09:46:48.474766 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 131/278] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:48 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:49.472316 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-28d2f714-990b-4124-bc63-4a06ce2f1679 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:49.474202 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-28d2f714-990b-4124-bc63-4a06ce2f1679 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:49.482591 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-28d2f714-990b-4124-bc63-4a06ce2f1679 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:49.483600 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-28d2f714-990b-4124-bc63-4a06ce2f1679 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:49.484065 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-28d2f714-990b-4124-bc63-4a06ce2f1679 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:49.484807 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-28d2f714-990b-4124-bc63-4a06ce2f1679 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:49.485613 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-28d2f714-990b-4124-bc63-4a06ce2f1679 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:49.486029 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-28d2f714-990b-4124-bc63-4a06ce2f1679 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:49.498790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8fd0a9ee-290e-4e8a-933b-97b7c2c3389a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:49.499542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8fd0a9ee-290e-4e8a-933b-97b7c2c3389a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:49.512038 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fd0a9ee-290e-4e8a-933b-97b7c2c3389a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:49.512548 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fd0a9ee-290e-4e8a-933b-97b7c2c3389a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:49.512995 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fd0a9ee-290e-4e8a-933b-97b7c2c3389a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:49.513730 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fd0a9ee-290e-4e8a-933b-97b7c2c3389a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:49.514228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fd0a9ee-290e-4e8a-933b-97b7c2c3389a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:49.514681 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fd0a9ee-290e-4e8a-933b-97b7c2c3389a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:49.526532 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=100 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:49.758905 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-28d2f714-990b-4124-bc63-4a06ce2f1679 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:49.770761 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-28d2f714-990b-4124-bc63-4a06ce2f1679 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.302919 Oct 28 09:46:49.771704 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 148/279] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:49 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:49.775545 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8fd0a9ee-290e-4e8a-933b-97b7c2c3389a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:49.787968 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8fd0a9ee-290e-4e8a-933b-97b7c2c3389a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.289680 Oct 28 09:46:49.790543 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 132/280] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:49 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:50.796541 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-0ed671f6-55ec-462f-aba8-2bb6e55fb0f2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:50.797911 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-0ed671f6-55ec-462f-aba8-2bb6e55fb0f2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:50.800208 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=2 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:50.801163 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=54,nova_api:INSERT=3 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:50.807478 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ed671f6-55ec-462f-aba8-2bb6e55fb0f2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:50.807968 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ed671f6-55ec-462f-aba8-2bb6e55fb0f2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:50.808401 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ed671f6-55ec-462f-aba8-2bb6e55fb0f2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:50.809152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ed671f6-55ec-462f-aba8-2bb6e55fb0f2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:50.809604 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ed671f6-55ec-462f-aba8-2bb6e55fb0f2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:50.810183 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ed671f6-55ec-462f-aba8-2bb6e55fb0f2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:50.815617 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-17f863a6-43d0-40c7-b3c2-beca9686f571 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:50.816224 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-17f863a6-43d0-40c7-b3c2-beca9686f571 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:50.818822 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=4 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:50.825484 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-17f863a6-43d0-40c7-b3c2-beca9686f571 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:50.825645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-17f863a6-43d0-40c7-b3c2-beca9686f571 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:50.825960 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-17f863a6-43d0-40c7-b3c2-beca9686f571 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:50.826680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-17f863a6-43d0-40c7-b3c2-beca9686f571 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:50.827130 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-17f863a6-43d0-40c7-b3c2-beca9686f571 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:50.827618 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-17f863a6-43d0-40c7-b3c2-beca9686f571 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:50.944334 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=8 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:51.081150 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0ed671f6-55ec-462f-aba8-2bb6e55fb0f2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:51.086438 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-17f863a6-43d0-40c7-b3c2-beca9686f571 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:51.096674 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-17f863a6-43d0-40c7-b3c2-beca9686f571 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.285313 Oct 28 09:46:51.105775 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 133/281] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:50 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:51.106017 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-0ed671f6-55ec-462f-aba8-2bb6e55fb0f2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.312439 Oct 28 09:46:51.106017 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 149/282] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:50 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:52.128783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f5a485b9-4c8d-4bc9-bccc-19b42f0b151e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:52.130444 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f5a485b9-4c8d-4bc9-bccc-19b42f0b151e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:52.132368 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=53,nova_api:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:52.139380 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-a79ac1f8-b355-4a9b-ad07-b4b61de40970 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:52.140497 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-a79ac1f8-b355-4a9b-ad07-b4b61de40970 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:52.140633 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f5a485b9-4c8d-4bc9-bccc-19b42f0b151e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:52.141793 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f5a485b9-4c8d-4bc9-bccc-19b42f0b151e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:52.142551 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f5a485b9-4c8d-4bc9-bccc-19b42f0b151e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:52.145416 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f5a485b9-4c8d-4bc9-bccc-19b42f0b151e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:52.145941 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f5a485b9-4c8d-4bc9-bccc-19b42f0b151e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:52.146380 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f5a485b9-4c8d-4bc9-bccc-19b42f0b151e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:52.151615 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a79ac1f8-b355-4a9b-ad07-b4b61de40970 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:52.154339 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a79ac1f8-b355-4a9b-ad07-b4b61de40970 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:52.154831 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a79ac1f8-b355-4a9b-ad07-b4b61de40970 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:52.160204 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a79ac1f8-b355-4a9b-ad07-b4b61de40970 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:52.161409 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a79ac1f8-b355-4a9b-ad07-b4b61de40970 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:52.161878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a79ac1f8-b355-4a9b-ad07-b4b61de40970 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:52.638129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-a79ac1f8-b355-4a9b-ad07-b4b61de40970 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:52.644645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f5a485b9-4c8d-4bc9-bccc-19b42f0b151e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:52.652455 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-a79ac1f8-b355-4a9b-ad07-b4b61de40970 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.517126 Oct 28 09:46:52.653203 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f5a485b9-4c8d-4bc9-bccc-19b42f0b151e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.528475 Oct 28 09:46:52.656943 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 150/283] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:52 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 523 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:52.667646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 134/284] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:52 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:53.680241 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-61e721ba-87a4-4f1f-94ac-9d98d61795e6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:53.680828 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-61e721ba-87a4-4f1f-94ac-9d98d61795e6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:53.686374 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a1285336-96ef-4d73-8606-dc50744e04ce tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:53.686904 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61e721ba-87a4-4f1f-94ac-9d98d61795e6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:53.687125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a1285336-96ef-4d73-8606-dc50744e04ce tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:53.687359 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61e721ba-87a4-4f1f-94ac-9d98d61795e6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:53.687786 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61e721ba-87a4-4f1f-94ac-9d98d61795e6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:53.688461 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61e721ba-87a4-4f1f-94ac-9d98d61795e6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:53.688925 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61e721ba-87a4-4f1f-94ac-9d98d61795e6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:53.689319 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-61e721ba-87a4-4f1f-94ac-9d98d61795e6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:53.693199 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a1285336-96ef-4d73-8606-dc50744e04ce tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:53.693674 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a1285336-96ef-4d73-8606-dc50744e04ce tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:53.694100 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a1285336-96ef-4d73-8606-dc50744e04ce tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:53.694804 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a1285336-96ef-4d73-8606-dc50744e04ce tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:53.695249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a1285336-96ef-4d73-8606-dc50744e04ce tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:53.695669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a1285336-96ef-4d73-8606-dc50744e04ce tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:53.874516 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-61e721ba-87a4-4f1f-94ac-9d98d61795e6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:53.881202 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a1285336-96ef-4d73-8606-dc50744e04ce tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:53.883284 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-61e721ba-87a4-4f1f-94ac-9d98d61795e6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.207469 Oct 28 09:46:53.883853 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 151/285] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:53 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:53.888792 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a1285336-96ef-4d73-8606-dc50744e04ce tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.206169 Oct 28 09:46:53.889478 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 135/286] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:53 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:54.906447 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b326e2e8-0e15-4f84-a50f-ff7d4e7b28eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:54.907044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b326e2e8-0e15-4f84-a50f-ff7d4e7b28eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:54.911741 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4e91ce1d-8786-414d-b099-00f2faf487fa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:54.912344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4e91ce1d-8786-414d-b099-00f2faf487fa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:54.913030 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b326e2e8-0e15-4f84-a50f-ff7d4e7b28eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:54.913512 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b326e2e8-0e15-4f84-a50f-ff7d4e7b28eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:54.913941 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b326e2e8-0e15-4f84-a50f-ff7d4e7b28eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:54.914662 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b326e2e8-0e15-4f84-a50f-ff7d4e7b28eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:54.915112 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b326e2e8-0e15-4f84-a50f-ff7d4e7b28eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:54.915530 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b326e2e8-0e15-4f84-a50f-ff7d4e7b28eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:54.918867 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e91ce1d-8786-414d-b099-00f2faf487fa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:54.919343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e91ce1d-8786-414d-b099-00f2faf487fa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:54.919771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e91ce1d-8786-414d-b099-00f2faf487fa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:54.920458 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e91ce1d-8786-414d-b099-00f2faf487fa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:54.920902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e91ce1d-8786-414d-b099-00f2faf487fa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:54.921322 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e91ce1d-8786-414d-b099-00f2faf487fa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:55.109268 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b326e2e8-0e15-4f84-a50f-ff7d4e7b28eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:55.112595 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-4e91ce1d-8786-414d-b099-00f2faf487fa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:55.116962 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b326e2e8-0e15-4f84-a50f-ff7d4e7b28eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.214461 Oct 28 09:46:55.117530 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 152/287] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:54 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:55.120720 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4e91ce1d-8786-414d-b099-00f2faf487fa tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.212731 Oct 28 09:46:55.121368 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 136/288] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:54 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:56.139260 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c27c0492-4f55-459c-8f96-27f4ddfe9af9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:56.144536 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b466cc0a-6ac9-4bda-9583-b0c1d0cc7260 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:56.145820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c27c0492-4f55-459c-8f96-27f4ddfe9af9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:56.146119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b466cc0a-6ac9-4bda-9583-b0c1d0cc7260 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:56.152273 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c27c0492-4f55-459c-8f96-27f4ddfe9af9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:56.153092 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c27c0492-4f55-459c-8f96-27f4ddfe9af9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:56.153386 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c27c0492-4f55-459c-8f96-27f4ddfe9af9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:56.154221 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c27c0492-4f55-459c-8f96-27f4ddfe9af9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:56.154872 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c27c0492-4f55-459c-8f96-27f4ddfe9af9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:56.155389 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c27c0492-4f55-459c-8f96-27f4ddfe9af9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:56.155828 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b466cc0a-6ac9-4bda-9583-b0c1d0cc7260 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:56.156368 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b466cc0a-6ac9-4bda-9583-b0c1d0cc7260 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:56.157017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b466cc0a-6ac9-4bda-9583-b0c1d0cc7260 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:56.157777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b466cc0a-6ac9-4bda-9583-b0c1d0cc7260 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:56.158442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b466cc0a-6ac9-4bda-9583-b0c1d0cc7260 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:56.159080 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b466cc0a-6ac9-4bda-9583-b0c1d0cc7260 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:56.462124 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c27c0492-4f55-459c-8f96-27f4ddfe9af9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:56.470678 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c27c0492-4f55-459c-8f96-27f4ddfe9af9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.335165 Oct 28 09:46:56.470892 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b466cc0a-6ac9-4bda-9583-b0c1d0cc7260 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:56.481249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b466cc0a-6ac9-4bda-9583-b0c1d0cc7260 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.340186 Oct 28 09:46:56.482183 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 137/289] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:56 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:56.487099 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 153/290] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:56 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:56.501335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-0470dae0-453a-45ba-90c9-949c29c9cab7 req-93e4854d-ebe5-45c1-8000-83c2eea6a505 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "04eedf41-b3f2-4b33-b524-988daff99fdc", "tag": "0ec06b02-a12d-496d-a40d-c56118810a83"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:46:56.512289 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-0470dae0-453a-45ba-90c9-949c29c9cab7 req-93e4854d-ebe5-45c1-8000-83c2eea6a505 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:56.512762 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-0470dae0-453a-45ba-90c9-949c29c9cab7 req-93e4854d-ebe5-45c1-8000-83c2eea6a505 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:56.513186 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-0470dae0-453a-45ba-90c9-949c29c9cab7 req-93e4854d-ebe5-45c1-8000-83c2eea6a505 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:56.525678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-0470dae0-453a-45ba-90c9-949c29c9cab7 req-93e4854d-ebe5-45c1-8000-83c2eea6a505 service nova] Creating event network-changed:0ec06b02-a12d-496d-a40d-c56118810a83 for instance 04eedf41-b3f2-4b33-b524-988daff99fdc on hv-cinder-zed-3 Oct 28 09:46:56.530612 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-0470dae0-453a-45ba-90c9-949c29c9cab7 req-93e4854d-ebe5-45c1-8000-83c2eea6a505 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047004 Oct 28 09:46:56.531208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 138/291] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:46:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:46:57.506721 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5d36b0f7-b910-4ca3-bf67-cbba5561a48c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:57.507255 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5d36b0f7-b910-4ca3-bf67-cbba5561a48c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:57.509325 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d6712e1b-4b86-4a71-bb69-475449e0beb9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:57.510029 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d6712e1b-4b86-4a71-bb69-475449e0beb9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:57.513262 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5d36b0f7-b910-4ca3-bf67-cbba5561a48c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:57.513851 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5d36b0f7-b910-4ca3-bf67-cbba5561a48c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:57.514231 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5d36b0f7-b910-4ca3-bf67-cbba5561a48c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:57.514897 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5d36b0f7-b910-4ca3-bf67-cbba5561a48c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:57.515345 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5d36b0f7-b910-4ca3-bf67-cbba5561a48c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:57.515820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5d36b0f7-b910-4ca3-bf67-cbba5561a48c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:57.517093 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6712e1b-4b86-4a71-bb69-475449e0beb9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:57.517569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6712e1b-4b86-4a71-bb69-475449e0beb9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:57.518004 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6712e1b-4b86-4a71-bb69-475449e0beb9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:57.518713 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6712e1b-4b86-4a71-bb69-475449e0beb9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:57.519156 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6712e1b-4b86-4a71-bb69-475449e0beb9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:57.519570 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6712e1b-4b86-4a71-bb69-475449e0beb9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:57.726636 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5d36b0f7-b910-4ca3-bf67-cbba5561a48c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:57.731199 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d6712e1b-4b86-4a71-bb69-475449e0beb9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:57.735252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5d36b0f7-b910-4ca3-bf67-cbba5561a48c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.232530 Oct 28 09:46:57.742307 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d6712e1b-4b86-4a71-bb69-475449e0beb9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.236877 Oct 28 09:46:57.742926 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 139/292] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:57 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:57.750822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 154/293] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:57 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:58.761279 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a6213d4d-ee7e-4264-bc6f-99ce7d311890 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:58.761811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a6213d4d-ee7e-4264-bc6f-99ce7d311890 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:58.765784 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-853d2e13-8e3c-4aec-9c1d-747562dfb355 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:46:58.766671 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-853d2e13-8e3c-4aec-9c1d-747562dfb355 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:46:58.768456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6213d4d-ee7e-4264-bc6f-99ce7d311890 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:58.768931 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6213d4d-ee7e-4264-bc6f-99ce7d311890 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:58.769353 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6213d4d-ee7e-4264-bc6f-99ce7d311890 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:58.770037 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6213d4d-ee7e-4264-bc6f-99ce7d311890 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:58.770510 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6213d4d-ee7e-4264-bc6f-99ce7d311890 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:58.770926 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6213d4d-ee7e-4264-bc6f-99ce7d311890 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:58.772829 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-853d2e13-8e3c-4aec-9c1d-747562dfb355 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:58.773399 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-853d2e13-8e3c-4aec-9c1d-747562dfb355 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:58.773824 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-853d2e13-8e3c-4aec-9c1d-747562dfb355 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:58.774600 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-853d2e13-8e3c-4aec-9c1d-747562dfb355 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:46:58.775052 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-853d2e13-8e3c-4aec-9c1d-747562dfb355 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:46:58.775462 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-853d2e13-8e3c-4aec-9c1d-747562dfb355 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:46:58.967349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a6213d4d-ee7e-4264-bc6f-99ce7d311890 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:58.972353 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-853d2e13-8e3c-4aec-9c1d-747562dfb355 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:46:58.975775 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a6213d4d-ee7e-4264-bc6f-99ce7d311890 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.218387 Oct 28 09:46:58.976639 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 140/294] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:58 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:46:58.981419 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-853d2e13-8e3c-4aec-9c1d-747562dfb355 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.219430 Oct 28 09:46:58.982032 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 155/295] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:58 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:46:59.999988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ad8b7a19-4045-4f70-b10a-7f322dafce46 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:00.000596 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ad8b7a19-4045-4f70-b10a-7f322dafce46 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:00.004666 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d20ab8cb-f9fa-482d-be98-219a264eb564 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:00.005277 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d20ab8cb-f9fa-482d-be98-219a264eb564 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:00.007256 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad8b7a19-4045-4f70-b10a-7f322dafce46 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:00.007743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad8b7a19-4045-4f70-b10a-7f322dafce46 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:00.008204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad8b7a19-4045-4f70-b10a-7f322dafce46 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:00.008894 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad8b7a19-4045-4f70-b10a-7f322dafce46 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:00.009341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad8b7a19-4045-4f70-b10a-7f322dafce46 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:00.009758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad8b7a19-4045-4f70-b10a-7f322dafce46 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:00.011130 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d20ab8cb-f9fa-482d-be98-219a264eb564 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:00.011611 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d20ab8cb-f9fa-482d-be98-219a264eb564 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:00.012050 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d20ab8cb-f9fa-482d-be98-219a264eb564 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:00.012733 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d20ab8cb-f9fa-482d-be98-219a264eb564 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:00.013189 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d20ab8cb-f9fa-482d-be98-219a264eb564 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:00.013604 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d20ab8cb-f9fa-482d-be98-219a264eb564 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:00.198725 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ad8b7a19-4045-4f70-b10a-7f322dafce46 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:00.205700 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d20ab8cb-f9fa-482d-be98-219a264eb564 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:00.206366 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ad8b7a19-4045-4f70-b10a-7f322dafce46 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.210240 Oct 28 09:47:00.206966 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 141/296] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:59 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:00.213971 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d20ab8cb-f9fa-482d-be98-219a264eb564 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.213115 Oct 28 09:47:00.214694 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 156/297] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:46:59 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:01.229991 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2ee9c2ee-e856-4b7d-9706-4d46dfa3cd29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:01.230535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2ee9c2ee-e856-4b7d-9706-4d46dfa3cd29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:01.236812 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-998ab9c7-1cbe-40c8-aa88-06499e5e6f5c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:01.237232 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ee9c2ee-e856-4b7d-9706-4d46dfa3cd29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:01.237445 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-998ab9c7-1cbe-40c8-aa88-06499e5e6f5c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:01.237615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ee9c2ee-e856-4b7d-9706-4d46dfa3cd29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:01.237938 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ee9c2ee-e856-4b7d-9706-4d46dfa3cd29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:01.238651 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ee9c2ee-e856-4b7d-9706-4d46dfa3cd29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:01.239107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ee9c2ee-e856-4b7d-9706-4d46dfa3cd29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:01.239527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ee9c2ee-e856-4b7d-9706-4d46dfa3cd29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:01.243552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-998ab9c7-1cbe-40c8-aa88-06499e5e6f5c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:01.244034 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-998ab9c7-1cbe-40c8-aa88-06499e5e6f5c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:01.244491 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-998ab9c7-1cbe-40c8-aa88-06499e5e6f5c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:01.245178 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-998ab9c7-1cbe-40c8-aa88-06499e5e6f5c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:01.245625 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-998ab9c7-1cbe-40c8-aa88-06499e5e6f5c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:01.246042 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-998ab9c7-1cbe-40c8-aa88-06499e5e6f5c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:01.264760 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:01.433017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2ee9c2ee-e856-4b7d-9706-4d46dfa3cd29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:01.441999 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-998ab9c7-1cbe-40c8-aa88-06499e5e6f5c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:01.444631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2ee9c2ee-e856-4b7d-9706-4d46dfa3cd29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.217750 Oct 28 09:47:01.445573 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 142/298] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:01 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:01.455868 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-998ab9c7-1cbe-40c8-aa88-06499e5e6f5c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.222388 Oct 28 09:47:01.456872 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 157/299] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:01 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:02.473773 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7430f47b-df8b-49b5-96b2-36d09b19228b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:02.474462 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7430f47b-df8b-49b5-96b2-36d09b19228b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:02.479546 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-aa27dee8-9bbf-47a3-9474-ef925e19c36e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:02.480360 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-aa27dee8-9bbf-47a3-9474-ef925e19c36e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:02.480687 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7430f47b-df8b-49b5-96b2-36d09b19228b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:02.481166 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7430f47b-df8b-49b5-96b2-36d09b19228b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:02.481611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7430f47b-df8b-49b5-96b2-36d09b19228b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:02.482282 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7430f47b-df8b-49b5-96b2-36d09b19228b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:02.482755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7430f47b-df8b-49b5-96b2-36d09b19228b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:02.483508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7430f47b-df8b-49b5-96b2-36d09b19228b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:02.486756 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aa27dee8-9bbf-47a3-9474-ef925e19c36e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:02.487241 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aa27dee8-9bbf-47a3-9474-ef925e19c36e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:02.487857 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aa27dee8-9bbf-47a3-9474-ef925e19c36e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:02.489095 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aa27dee8-9bbf-47a3-9474-ef925e19c36e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:02.489508 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aa27dee8-9bbf-47a3-9474-ef925e19c36e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:02.490122 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aa27dee8-9bbf-47a3-9474-ef925e19c36e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:02.700041 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7430f47b-df8b-49b5-96b2-36d09b19228b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:02.708388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7430f47b-df8b-49b5-96b2-36d09b19228b tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.238631 Oct 28 09:47:02.709125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 143/300] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:02 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:02.711741 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-aa27dee8-9bbf-47a3-9474-ef925e19c36e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:02.719577 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-aa27dee8-9bbf-47a3-9474-ef925e19c36e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.243857 Oct 28 09:47:02.720120 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 158/301] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:02 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:03.731502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-626687db-04ee-4a5c-8410-0d27cdfac22e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:03.732127 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-626687db-04ee-4a5c-8410-0d27cdfac22e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:03.739233 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-626687db-04ee-4a5c-8410-0d27cdfac22e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:03.739706 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-626687db-04ee-4a5c-8410-0d27cdfac22e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:03.740238 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-626687db-04ee-4a5c-8410-0d27cdfac22e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:03.740912 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-626687db-04ee-4a5c-8410-0d27cdfac22e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:03.741361 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-626687db-04ee-4a5c-8410-0d27cdfac22e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:03.741768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-626687db-04ee-4a5c-8410-0d27cdfac22e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:03.743864 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5fbae244-8644-415b-8a26-4fde6b93310f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:03.744484 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5fbae244-8644-415b-8a26-4fde6b93310f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:03.750308 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5fbae244-8644-415b-8a26-4fde6b93310f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:03.750848 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5fbae244-8644-415b-8a26-4fde6b93310f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:03.751277 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5fbae244-8644-415b-8a26-4fde6b93310f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:03.751975 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5fbae244-8644-415b-8a26-4fde6b93310f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:03.752423 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5fbae244-8644-415b-8a26-4fde6b93310f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:03.752838 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5fbae244-8644-415b-8a26-4fde6b93310f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:03.935616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-626687db-04ee-4a5c-8410-0d27cdfac22e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:03.939637 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5fbae244-8644-415b-8a26-4fde6b93310f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:03.944040 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-626687db-04ee-4a5c-8410-0d27cdfac22e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.216386 Oct 28 09:47:03.944618 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 144/302] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:03 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:03.948317 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5fbae244-8644-415b-8a26-4fde6b93310f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.208193 Oct 28 09:47:03.948959 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 159/303] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:03 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:04.968427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d0e8e5f9-e6b7-40bd-8d7b-2ec3f4ec5334 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:04.969161 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d0e8e5f9-e6b7-40bd-8d7b-2ec3f4ec5334 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:04.971261 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-19c8a4f0-96d2-4f8d-930f-5085411981d0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:04.971971 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-19c8a4f0-96d2-4f8d-930f-5085411981d0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:04.978160 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0e8e5f9-e6b7-40bd-8d7b-2ec3f4ec5334 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:04.978443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-19c8a4f0-96d2-4f8d-930f-5085411981d0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:04.978689 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0e8e5f9-e6b7-40bd-8d7b-2ec3f4ec5334 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:04.978933 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-19c8a4f0-96d2-4f8d-930f-5085411981d0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:04.979180 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0e8e5f9-e6b7-40bd-8d7b-2ec3f4ec5334 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:04.979364 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-19c8a4f0-96d2-4f8d-930f-5085411981d0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:04.979880 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0e8e5f9-e6b7-40bd-8d7b-2ec3f4ec5334 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:04.980050 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-19c8a4f0-96d2-4f8d-930f-5085411981d0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:04.980330 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0e8e5f9-e6b7-40bd-8d7b-2ec3f4ec5334 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:04.980499 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-19c8a4f0-96d2-4f8d-930f-5085411981d0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:04.980750 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0e8e5f9-e6b7-40bd-8d7b-2ec3f4ec5334 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:04.980919 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-19c8a4f0-96d2-4f8d-930f-5085411981d0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:05.175935 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d0e8e5f9-e6b7-40bd-8d7b-2ec3f4ec5334 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:05.180003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-19c8a4f0-96d2-4f8d-930f-5085411981d0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:05.183599 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d0e8e5f9-e6b7-40bd-8d7b-2ec3f4ec5334 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.219324 Oct 28 09:47:05.184167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 145/304] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:04 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:05.188284 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-19c8a4f0-96d2-4f8d-930f-5085411981d0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.220881 Oct 28 09:47:05.188996 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 160/305] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:04 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:06.207437 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f303e769-66a7-448a-bc39-72a45785a6cf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:06.208067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f303e769-66a7-448a-bc39-72a45785a6cf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:06.212243 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d3aee683-eff9-405e-8dd9-e767d348548c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:06.212852 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d3aee683-eff9-405e-8dd9-e767d348548c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:06.214581 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f303e769-66a7-448a-bc39-72a45785a6cf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:06.215055 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f303e769-66a7-448a-bc39-72a45785a6cf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:06.215479 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f303e769-66a7-448a-bc39-72a45785a6cf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:06.216185 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f303e769-66a7-448a-bc39-72a45785a6cf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:06.216628 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f303e769-66a7-448a-bc39-72a45785a6cf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:06.217060 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f303e769-66a7-448a-bc39-72a45785a6cf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:06.218727 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3aee683-eff9-405e-8dd9-e767d348548c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:06.219208 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3aee683-eff9-405e-8dd9-e767d348548c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:06.219668 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3aee683-eff9-405e-8dd9-e767d348548c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:06.220352 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3aee683-eff9-405e-8dd9-e767d348548c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:06.220825 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3aee683-eff9-405e-8dd9-e767d348548c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:06.221242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3aee683-eff9-405e-8dd9-e767d348548c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:06.421750 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f303e769-66a7-448a-bc39-72a45785a6cf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:06.427278 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d3aee683-eff9-405e-8dd9-e767d348548c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:06.429773 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f303e769-66a7-448a-bc39-72a45785a6cf tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.226174 Oct 28 09:47:06.430444 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 146/306] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:06 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:06.435391 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d3aee683-eff9-405e-8dd9-e767d348548c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.226903 Oct 28 09:47:06.436117 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 161/307] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:06 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:07.453534 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a44eb480-a556-409a-a1a0-1c0c11d7b0cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:07.454135 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a44eb480-a556-409a-a1a0-1c0c11d7b0cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:07.458991 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-41f1e437-48fa-422d-9da0-5bb4645e02e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:07.459631 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-41f1e437-48fa-422d-9da0-5bb4645e02e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:07.460671 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a44eb480-a556-409a-a1a0-1c0c11d7b0cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:07.461232 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a44eb480-a556-409a-a1a0-1c0c11d7b0cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:07.461570 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a44eb480-a556-409a-a1a0-1c0c11d7b0cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:07.462251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a44eb480-a556-409a-a1a0-1c0c11d7b0cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:07.462735 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a44eb480-a556-409a-a1a0-1c0c11d7b0cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:07.463146 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a44eb480-a556-409a-a1a0-1c0c11d7b0cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:07.465863 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-41f1e437-48fa-422d-9da0-5bb4645e02e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:07.466343 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-41f1e437-48fa-422d-9da0-5bb4645e02e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:07.466798 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-41f1e437-48fa-422d-9da0-5bb4645e02e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:07.467487 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-41f1e437-48fa-422d-9da0-5bb4645e02e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:07.467935 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-41f1e437-48fa-422d-9da0-5bb4645e02e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:07.468351 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-41f1e437-48fa-422d-9da0-5bb4645e02e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:07.662933 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-41f1e437-48fa-422d-9da0-5bb4645e02e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:07.666064 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a44eb480-a556-409a-a1a0-1c0c11d7b0cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:07.670945 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-41f1e437-48fa-422d-9da0-5bb4645e02e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.215664 Oct 28 09:47:07.671515 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 162/308] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:07 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:07.674419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a44eb480-a556-409a-a1a0-1c0c11d7b0cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.224742 Oct 28 09:47:07.675067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 147/309] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:07 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:08.695175 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6aef1c87-ba40-404c-a705-ee926c242a5f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:08.695859 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6aef1c87-ba40-404c-a705-ee926c242a5f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:08.700447 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5e792ce4-6c13-4f86-be48-5050a11a0cd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:08.701051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5e792ce4-6c13-4f86-be48-5050a11a0cd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:08.702672 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6aef1c87-ba40-404c-a705-ee926c242a5f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:08.703154 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6aef1c87-ba40-404c-a705-ee926c242a5f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:08.703612 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6aef1c87-ba40-404c-a705-ee926c242a5f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:08.704301 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6aef1c87-ba40-404c-a705-ee926c242a5f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:08.704775 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6aef1c87-ba40-404c-a705-ee926c242a5f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:08.705188 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6aef1c87-ba40-404c-a705-ee926c242a5f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:08.707935 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e792ce4-6c13-4f86-be48-5050a11a0cd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:08.708421 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e792ce4-6c13-4f86-be48-5050a11a0cd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:08.711395 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e792ce4-6c13-4f86-be48-5050a11a0cd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:08.712360 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e792ce4-6c13-4f86-be48-5050a11a0cd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:08.712823 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e792ce4-6c13-4f86-be48-5050a11a0cd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:08.713237 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e792ce4-6c13-4f86-be48-5050a11a0cd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:08.904448 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6aef1c87-ba40-404c-a705-ee926c242a5f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:08.907518 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5e792ce4-6c13-4f86-be48-5050a11a0cd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:08.912091 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6aef1c87-ba40-404c-a705-ee926c242a5f tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.221599 Oct 28 09:47:08.912658 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 163/310] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:08 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:08.915391 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5e792ce4-6c13-4f86-be48-5050a11a0cd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.220846 Oct 28 09:47:08.916022 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 148/311] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:08 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:09.936358 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c180e83f-5935-49db-9c95-632c57d41343 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:09.936926 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c180e83f-5935-49db-9c95-632c57d41343 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:09.939054 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-dcb01c96-6ca2-440c-9ed9-cbd70f9f46e0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:09.939672 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-dcb01c96-6ca2-440c-9ed9-cbd70f9f46e0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:09.944038 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c180e83f-5935-49db-9c95-632c57d41343 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:09.944516 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c180e83f-5935-49db-9c95-632c57d41343 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:09.944943 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c180e83f-5935-49db-9c95-632c57d41343 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:09.945627 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c180e83f-5935-49db-9c95-632c57d41343 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:09.946070 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c180e83f-5935-49db-9c95-632c57d41343 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:09.946504 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c180e83f-5935-49db-9c95-632c57d41343 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:09.946691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dcb01c96-6ca2-440c-9ed9-cbd70f9f46e0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:09.947169 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dcb01c96-6ca2-440c-9ed9-cbd70f9f46e0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:09.947597 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dcb01c96-6ca2-440c-9ed9-cbd70f9f46e0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:09.948282 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dcb01c96-6ca2-440c-9ed9-cbd70f9f46e0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:09.948723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dcb01c96-6ca2-440c-9ed9-cbd70f9f46e0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:09.949141 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dcb01c96-6ca2-440c-9ed9-cbd70f9f46e0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:09.969488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=100 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:10.145613 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c180e83f-5935-49db-9c95-632c57d41343 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:10.148820 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-dcb01c96-6ca2-440c-9ed9-cbd70f9f46e0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:10.154179 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c180e83f-5935-49db-9c95-632c57d41343 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.221815 Oct 28 09:47:10.154789 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 164/312] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:09 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:10.157356 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-dcb01c96-6ca2-440c-9ed9-cbd70f9f46e0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.222652 Oct 28 09:47:10.157982 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 149/313] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:09 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:10.915543 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-764bb319-09e4-499f-b5b3-02d0fbc09811 req-4b2e62b9-dd5d-4de7-a5b0-743c8c05ca69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "04eedf41-b3f2-4b33-b524-988daff99fdc", "name": "network-vif-plugged", "status": "completed", "tag": "0ec06b02-a12d-496d-a40d-c56118810a83"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:10.926344 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-764bb319-09e4-499f-b5b3-02d0fbc09811 req-4b2e62b9-dd5d-4de7-a5b0-743c8c05ca69 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:10.926868 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-764bb319-09e4-499f-b5b3-02d0fbc09811 req-4b2e62b9-dd5d-4de7-a5b0-743c8c05ca69 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:10.927287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-764bb319-09e4-499f-b5b3-02d0fbc09811 req-4b2e62b9-dd5d-4de7-a5b0-743c8c05ca69 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:10.940211 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-764bb319-09e4-499f-b5b3-02d0fbc09811 req-4b2e62b9-dd5d-4de7-a5b0-743c8c05ca69 service nova] Creating event network-vif-plugged:0ec06b02-a12d-496d-a40d-c56118810a83 for instance 04eedf41-b3f2-4b33-b524-988daff99fdc on hv-cinder-zed-3 Oct 28 09:47:10.945975 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-764bb319-09e4-499f-b5b3-02d0fbc09811 req-4b2e62b9-dd5d-4de7-a5b0-743c8c05ca69 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034539 Oct 28 09:47:10.946763 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 165/314] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:47:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:11.178285 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-19c832ae-2d0d-447b-a071-9c86bfb02bca tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:11.179390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-19c832ae-2d0d-447b-a071-9c86bfb02bca tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:11.182067 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3e873b2b-6d58-496c-98f6-8d8b106996f1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:11.184035 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-3e873b2b-6d58-496c-98f6-8d8b106996f1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:11.188310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-19c832ae-2d0d-447b-a071-9c86bfb02bca tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:11.188867 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-19c832ae-2d0d-447b-a071-9c86bfb02bca tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:11.189712 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-19c832ae-2d0d-447b-a071-9c86bfb02bca tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:11.191606 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-19c832ae-2d0d-447b-a071-9c86bfb02bca tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:11.191606 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-19c832ae-2d0d-447b-a071-9c86bfb02bca tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:11.191924 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3e873b2b-6d58-496c-98f6-8d8b106996f1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:11.192178 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-19c832ae-2d0d-447b-a071-9c86bfb02bca tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:11.192270 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3e873b2b-6d58-496c-98f6-8d8b106996f1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:11.192270 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3e873b2b-6d58-496c-98f6-8d8b106996f1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:11.193192 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3e873b2b-6d58-496c-98f6-8d8b106996f1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:11.193830 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3e873b2b-6d58-496c-98f6-8d8b106996f1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:11.194276 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3e873b2b-6d58-496c-98f6-8d8b106996f1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:11.426697 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-19c832ae-2d0d-447b-a071-9c86bfb02bca tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:11.431082 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3e873b2b-6d58-496c-98f6-8d8b106996f1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:11.434479 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-19c832ae-2d0d-447b-a071-9c86bfb02bca tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.260598 Oct 28 09:47:11.435046 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 150/315] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:11 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:11.439096 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-3e873b2b-6d58-496c-98f6-8d8b106996f1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.261503 Oct 28 09:47:11.439650 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 166/316] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:11 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:12.458953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-446b99b8-bc7f-4816-8f86-d75861689d7e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:12.459554 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-446b99b8-bc7f-4816-8f86-d75861689d7e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:12.461406 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c4c0e603-e523-45a9-b655-8e6fad5e5be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:12.462038 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c4c0e603-e523-45a9-b655-8e6fad5e5be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:12.465962 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-446b99b8-bc7f-4816-8f86-d75861689d7e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:12.466476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-446b99b8-bc7f-4816-8f86-d75861689d7e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:12.466924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-446b99b8-bc7f-4816-8f86-d75861689d7e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:12.467588 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-446b99b8-bc7f-4816-8f86-d75861689d7e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:12.467976 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c4c0e603-e523-45a9-b655-8e6fad5e5be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:12.468079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-446b99b8-bc7f-4816-8f86-d75861689d7e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:12.468427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-446b99b8-bc7f-4816-8f86-d75861689d7e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:12.468547 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c4c0e603-e523-45a9-b655-8e6fad5e5be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:12.468871 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c4c0e603-e523-45a9-b655-8e6fad5e5be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:12.469582 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c4c0e603-e523-45a9-b655-8e6fad5e5be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:12.470022 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c4c0e603-e523-45a9-b655-8e6fad5e5be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:12.470447 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c4c0e603-e523-45a9-b655-8e6fad5e5be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:12.689615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-446b99b8-bc7f-4816-8f86-d75861689d7e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:12.696040 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c4c0e603-e523-45a9-b655-8e6fad5e5be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:12.697907 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-446b99b8-bc7f-4816-8f86-d75861689d7e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.243462 Oct 28 09:47:12.698497 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 151/317] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:12 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:12.703972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c4c0e603-e523-45a9-b655-8e6fad5e5be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.246952 Oct 28 09:47:12.704704 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 167/318] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:12 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:13.721082 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-af086967-ca9e-43a1-9202-55c47e56e839 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:13.721774 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-af086967-ca9e-43a1-9202-55c47e56e839 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:13.726822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-11d81e43-a961-43d2-9394-c6a73439f994 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:13.727905 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-11d81e43-a961-43d2-9394-c6a73439f994 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:13.728036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-af086967-ca9e-43a1-9202-55c47e56e839 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:13.728417 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-af086967-ca9e-43a1-9202-55c47e56e839 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:13.728854 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-af086967-ca9e-43a1-9202-55c47e56e839 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:13.729540 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-af086967-ca9e-43a1-9202-55c47e56e839 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:13.729985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-af086967-ca9e-43a1-9202-55c47e56e839 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:13.730424 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-af086967-ca9e-43a1-9202-55c47e56e839 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:13.733759 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-11d81e43-a961-43d2-9394-c6a73439f994 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:13.734238 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-11d81e43-a961-43d2-9394-c6a73439f994 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:13.734698 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-11d81e43-a961-43d2-9394-c6a73439f994 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:13.735382 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-11d81e43-a961-43d2-9394-c6a73439f994 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:13.735826 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-11d81e43-a961-43d2-9394-c6a73439f994 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:13.736237 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-11d81e43-a961-43d2-9394-c6a73439f994 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:13.928768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-af086967-ca9e-43a1-9202-55c47e56e839 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:13.931786 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-11d81e43-a961-43d2-9394-c6a73439f994 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:13.938428 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-af086967-ca9e-43a1-9202-55c47e56e839 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.220764 Oct 28 09:47:13.939511 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 152/319] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:13 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:13.939734 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-11d81e43-a961-43d2-9394-c6a73439f994 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.216481 Oct 28 09:47:13.940099 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 168/320] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:13 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:14.963625 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-f637c746-2f89-4ec1-b16c-131d157fb227 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:14.964235 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-f637c746-2f89-4ec1-b16c-131d157fb227 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:14.965518 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0375a602-4147-4d4a-b31f-8485777c3def tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:14.966177 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0375a602-4147-4d4a-b31f-8485777c3def tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:14.970331 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f637c746-2f89-4ec1-b16c-131d157fb227 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:14.970850 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f637c746-2f89-4ec1-b16c-131d157fb227 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:14.971279 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f637c746-2f89-4ec1-b16c-131d157fb227 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:14.971970 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f637c746-2f89-4ec1-b16c-131d157fb227 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:14.972416 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f637c746-2f89-4ec1-b16c-131d157fb227 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:14.972887 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0375a602-4147-4d4a-b31f-8485777c3def tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:14.973517 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f637c746-2f89-4ec1-b16c-131d157fb227 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:14.973878 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0375a602-4147-4d4a-b31f-8485777c3def tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:14.974305 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0375a602-4147-4d4a-b31f-8485777c3def tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:14.975014 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0375a602-4147-4d4a-b31f-8485777c3def tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:14.975458 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0375a602-4147-4d4a-b31f-8485777c3def tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:14.975878 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0375a602-4147-4d4a-b31f-8485777c3def tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:15.161376 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-f637c746-2f89-4ec1-b16c-131d157fb227 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:15.169177 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-f637c746-2f89-4ec1-b16c-131d157fb227 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1358 microversion: 2.1 time: 0.209787 Oct 28 09:47:15.169939 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 169/321] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:14 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1358 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:15.173266 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0375a602-4147-4d4a-b31f-8485777c3def tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:15.181084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0375a602-4147-4d4a-b31f-8485777c3def tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.221834 Oct 28 09:47:15.181716 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 153/322] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:14 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:16.054258 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b0234513-282e-4fc4-8b1d-b562ffae8939 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:16.054820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b0234513-282e-4fc4-8b1d-b562ffae8939 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:16.061308 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b0234513-282e-4fc4-8b1d-b562ffae8939 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:16.061811 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b0234513-282e-4fc4-8b1d-b562ffae8939 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:16.062266 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b0234513-282e-4fc4-8b1d-b562ffae8939 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:16.062972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b0234513-282e-4fc4-8b1d-b562ffae8939 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:16.063419 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b0234513-282e-4fc4-8b1d-b562ffae8939 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:16.063832 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b0234513-282e-4fc4-8b1d-b562ffae8939 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:16.087969 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b0234513-282e-4fc4-8b1d-b562ffae8939 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/ips" status: 200 len: 151 microversion: 2.20 time: 0.037588 Oct 28 09:47:16.088700 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 170/323] 10.222.0.25 () {60 vars in 1289 bytes} [Fri Oct 28 09:47:16 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/ips => generated 151 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:47:16.193747 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a1997e85-8cfa-4c2f-b737-a100743cbf61 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:16.194977 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a1997e85-8cfa-4c2f-b737-a100743cbf61 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:16.202110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a1997e85-8cfa-4c2f-b737-a100743cbf61 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:16.202656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a1997e85-8cfa-4c2f-b737-a100743cbf61 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:16.203277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a1997e85-8cfa-4c2f-b737-a100743cbf61 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:16.204112 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a1997e85-8cfa-4c2f-b737-a100743cbf61 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:16.204715 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a1997e85-8cfa-4c2f-b737-a100743cbf61 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:16.204899 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c34c50c7-caf5-45e3-924b-4531260b8141 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:16.205285 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a1997e85-8cfa-4c2f-b737-a100743cbf61 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:16.205846 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c34c50c7-caf5-45e3-924b-4531260b8141 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:16.227753 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c34c50c7-caf5-45e3-924b-4531260b8141 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:16.228472 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c34c50c7-caf5-45e3-924b-4531260b8141 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:16.229092 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c34c50c7-caf5-45e3-924b-4531260b8141 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:16.229950 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c34c50c7-caf5-45e3-924b-4531260b8141 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:16.230577 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c34c50c7-caf5-45e3-924b-4531260b8141 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:16.231147 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c34c50c7-caf5-45e3-924b-4531260b8141 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:16.402865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a1997e85-8cfa-4c2f-b737-a100743cbf61 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:16.416630 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a1997e85-8cfa-4c2f-b737-a100743cbf61 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1548 microversion: 2.1 time: 0.227660 Oct 28 09:47:16.417702 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 154/324] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:16 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1548 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:16.445063 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-83e28436-df3d-4b6f-9632-57881c77af7a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:16.445601 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-83e28436-df3d-4b6f-9632-57881c77af7a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:16.454732 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83e28436-df3d-4b6f-9632-57881c77af7a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:16.455465 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83e28436-df3d-4b6f-9632-57881c77af7a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:16.456085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83e28436-df3d-4b6f-9632-57881c77af7a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:16.457512 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83e28436-df3d-4b6f-9632-57881c77af7a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:16.458112 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83e28436-df3d-4b6f-9632-57881c77af7a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:16.458765 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83e28436-df3d-4b6f-9632-57881c77af7a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:16.480215 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c34c50c7-caf5-45e3-924b-4531260b8141 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:16.495567 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c34c50c7-caf5-45e3-924b-4531260b8141 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.295021 Oct 28 09:47:16.496407 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 171/325] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:16 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:16.621798 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-83e28436-df3d-4b6f-9632-57881c77af7a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:16.632350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-83e28436-df3d-4b6f-9632-57881c77af7a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1548 microversion: 2.1 time: 0.191908 Oct 28 09:47:16.633099 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 155/326] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:16 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1548 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:16.655575 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-73b0bd36-f22d-4141-9de8-49a8a07c795d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:16.660440 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-73b0bd36-f22d-4141-9de8-49a8a07c795d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:16.661202 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-73b0bd36-f22d-4141-9de8-49a8a07c795d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Searching by: {'deleted': False, 'project_id': '6f99f17a6cab46039419eb3b44bdd142'} {{(pid=127029) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 28 09:47:16.668762 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73b0bd36-f22d-4141-9de8-49a8a07c795d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:16.669310 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73b0bd36-f22d-4141-9de8-49a8a07c795d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:16.669795 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73b0bd36-f22d-4141-9de8-49a8a07c795d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:16.670573 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73b0bd36-f22d-4141-9de8-49a8a07c795d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:16.671026 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73b0bd36-f22d-4141-9de8-49a8a07c795d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:16.671557 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73b0bd36-f22d-4141-9de8-49a8a07c795d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:16.782234 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.multi_cell_list [None req-73b0bd36-f22d-4141-9de8-49a8a07c795d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=127029) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 28 09:47:16.885318 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-73b0bd36-f22d-4141-9de8-49a8a07c795d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.233388 Oct 28 09:47:16.885895 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 172/327] 10.222.0.25 () {58 vars in 1125 bytes} [Fri Oct 28 09:47:16 2022] GET /compute/v2.1/servers => generated 343 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:16.898284 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1adae3f3-cd77-4eba-b145-4bbd6c17dac0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:16.898931 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1adae3f3-cd77-4eba-b145-4bbd6c17dac0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:16.904870 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1adae3f3-cd77-4eba-b145-4bbd6c17dac0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:16.905339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1adae3f3-cd77-4eba-b145-4bbd6c17dac0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:16.905762 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1adae3f3-cd77-4eba-b145-4bbd6c17dac0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:16.906453 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1adae3f3-cd77-4eba-b145-4bbd6c17dac0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:16.906900 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1adae3f3-cd77-4eba-b145-4bbd6c17dac0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:16.907310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1adae3f3-cd77-4eba-b145-4bbd6c17dac0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:17.037759 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1adae3f3-cd77-4eba-b145-4bbd6c17dac0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:17.045755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1adae3f3-cd77-4eba-b145-4bbd6c17dac0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1548 microversion: 2.1 time: 0.151539 Oct 28 09:47:17.046309 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 156/328] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:16 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1548 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:17.521699 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8d31acd6-731f-49fc-9ff0-94e6b6b39327 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:17.522351 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8d31acd6-731f-49fc-9ff0-94e6b6b39327 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:17.528696 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8d31acd6-731f-49fc-9ff0-94e6b6b39327 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:17.529189 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8d31acd6-731f-49fc-9ff0-94e6b6b39327 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:17.529629 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8d31acd6-731f-49fc-9ff0-94e6b6b39327 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:17.530339 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8d31acd6-731f-49fc-9ff0-94e6b6b39327 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:17.530828 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8d31acd6-731f-49fc-9ff0-94e6b6b39327 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:17.531339 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8d31acd6-731f-49fc-9ff0-94e6b6b39327 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:17.697354 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-8d31acd6-731f-49fc-9ff0-94e6b6b39327 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:17.705194 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8d31acd6-731f-49fc-9ff0-94e6b6b39327 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.188228 Oct 28 09:47:17.705770 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 173/329] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:17 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:18.009612 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9827fe76-0da2-4929-ac02-2be0effcc7d2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:18.010159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9827fe76-0da2-4929-ac02-2be0effcc7d2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:18.020089 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9827fe76-0da2-4929-ac02-2be0effcc7d2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:18.020642 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9827fe76-0da2-4929-ac02-2be0effcc7d2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:18.021046 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9827fe76-0da2-4929-ac02-2be0effcc7d2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:18.024763 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9827fe76-0da2-4929-ac02-2be0effcc7d2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:18.025276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9827fe76-0da2-4929-ac02-2be0effcc7d2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:18.025725 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9827fe76-0da2-4929-ac02-2be0effcc7d2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:18.063132 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9827fe76-0da2-4929-ac02-2be0effcc7d2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/ips" status: 200 len: 145 microversion: 2.1 time: 0.056879 Oct 28 09:47:18.064308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 157/330] 10.222.0.25 () {58 vars in 1248 bytes} [Fri Oct 28 09:47:18 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/ips => generated 145 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:18.538737 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5e89fe8c-3491-4b98-b87d-65d00b535dc6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:18.539859 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5e89fe8c-3491-4b98-b87d-65d00b535dc6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:18.546721 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e89fe8c-3491-4b98-b87d-65d00b535dc6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:18.547499 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e89fe8c-3491-4b98-b87d-65d00b535dc6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:18.548016 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e89fe8c-3491-4b98-b87d-65d00b535dc6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:18.548741 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e89fe8c-3491-4b98-b87d-65d00b535dc6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:18.549164 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e89fe8c-3491-4b98-b87d-65d00b535dc6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:18.549746 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e89fe8c-3491-4b98-b87d-65d00b535dc6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:18.580235 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.objects.instance [None req-5e89fe8c-3491-4b98-b87d-65d00b535dc6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lazy-loading 'flavor' on Instance uuid c716d5fd-f257-4f6a-8f52-18ad59e5c31e {{(pid=127029) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:47:18.585001 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:18.650474 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-78e9339f-b888-49a2-ba8a-decd34449421 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4bfce979-c3b3-4e9d-b857-326102f2c9ca"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:18.653763 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-78e9339f-b888-49a2-ba8a-decd34449421 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:18.661447 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78e9339f-b888-49a2-ba8a-decd34449421 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:18.661933 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78e9339f-b888-49a2-ba8a-decd34449421 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:18.662359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78e9339f-b888-49a2-ba8a-decd34449421 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:18.663237 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78e9339f-b888-49a2-ba8a-decd34449421 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:18.663726 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78e9339f-b888-49a2-ba8a-decd34449421 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:18.664214 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78e9339f-b888-49a2-ba8a-decd34449421 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:18.817696 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5e89fe8c-3491-4b98-b87d-65d00b535dc6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "POST /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/action" status: 202 len: 0 microversion: 2.20 time: 0.282310 Oct 28 09:47:18.823129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 174/331] 10.222.0.25 () {62 vars in 1319 bytes} [Fri Oct 28 09:47:18 2022] POST /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/action => generated 0 bytes in 285 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:47:18.826320 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d9b59a3c-edfa-47f8-afbb-93803a63211e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:18.827048 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d9b59a3c-edfa-47f8-afbb-93803a63211e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:18.833799 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d9b59a3c-edfa-47f8-afbb-93803a63211e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:18.834288 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d9b59a3c-edfa-47f8-afbb-93803a63211e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:18.834778 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d9b59a3c-edfa-47f8-afbb-93803a63211e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:18.835514 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d9b59a3c-edfa-47f8-afbb-93803a63211e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:18.836115 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d9b59a3c-edfa-47f8-afbb-93803a63211e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:18.836579 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d9b59a3c-edfa-47f8-afbb-93803a63211e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:18.992462 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d9b59a3c-edfa-47f8-afbb-93803a63211e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:19.000424 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d9b59a3c-edfa-47f8-afbb-93803a63211e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.180360 Oct 28 09:47:19.001637 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 175/332] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:18 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:19.007750 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b14f4203-027f-4e11-b020-a05643fb7d2f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:19.008476 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b14f4203-027f-4e11-b020-a05643fb7d2f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:19.015467 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b14f4203-027f-4e11-b020-a05643fb7d2f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:19.016237 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b14f4203-027f-4e11-b020-a05643fb7d2f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:19.016891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b14f4203-027f-4e11-b020-a05643fb7d2f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:19.017805 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b14f4203-027f-4e11-b020-a05643fb7d2f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:19.019361 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b14f4203-027f-4e11-b020-a05643fb7d2f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:19.020014 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b14f4203-027f-4e11-b020-a05643fb7d2f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:19.168111 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b14f4203-027f-4e11-b020-a05643fb7d2f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:19.179672 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b14f4203-027f-4e11-b020-a05643fb7d2f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:19.181471 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b14f4203-027f-4e11-b020-a05643fb7d2f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:19.182850 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b14f4203-027f-4e11-b020-a05643fb7d2f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.180059 Oct 28 09:47:19.199099 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 176/333] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:19 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:19.410314 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-78e9339f-b888-49a2-ba8a-decd34449421 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "POST /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.763610 Oct 28 09:47:19.411322 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 158/334] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:47:18 2022] POST /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments => generated 194 bytes in 765 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:47:20.030354 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8047a34a-adb6-453d-a88a-08cdadd42909 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:20.031095 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8047a34a-adb6-453d-a88a-08cdadd42909 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:20.038552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8047a34a-adb6-453d-a88a-08cdadd42909 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:20.039240 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8047a34a-adb6-453d-a88a-08cdadd42909 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:20.039675 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8047a34a-adb6-453d-a88a-08cdadd42909 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:20.040370 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8047a34a-adb6-453d-a88a-08cdadd42909 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:20.041083 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8047a34a-adb6-453d-a88a-08cdadd42909 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:20.041686 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8047a34a-adb6-453d-a88a-08cdadd42909 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:20.197659 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-8047a34a-adb6-453d-a88a-08cdadd42909 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:20.206055 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8047a34a-adb6-453d-a88a-08cdadd42909 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.182291 Oct 28 09:47:20.206732 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 177/335] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:20 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:20.218579 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8fb805fd-423a-4846-bf3e-a022d1ef56c5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:20.218579 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8fb805fd-423a-4846-bf3e-a022d1ef56c5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:20.218579 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fb805fd-423a-4846-bf3e-a022d1ef56c5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:20.218859 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fb805fd-423a-4846-bf3e-a022d1ef56c5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:20.219295 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fb805fd-423a-4846-bf3e-a022d1ef56c5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:20.219991 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fb805fd-423a-4846-bf3e-a022d1ef56c5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:20.220471 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fb805fd-423a-4846-bf3e-a022d1ef56c5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:20.220891 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fb805fd-423a-4846-bf3e-a022d1ef56c5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:20.358719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8fb805fd-423a-4846-bf3e-a022d1ef56c5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:20.368232 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8fb805fd-423a-4846-bf3e-a022d1ef56c5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:20.370106 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8fb805fd-423a-4846-bf3e-a022d1ef56c5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:20.371592 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8fb805fd-423a-4846-bf3e-a022d1ef56c5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.165990 Oct 28 09:47:20.372183 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 159/336] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:20 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:47:20.858856 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-9e54f937-308e-49ce-ab0d-cf70a7b9a8fd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "505290f3-07f2-4f85-8d41-60f298323282"}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:20.861335 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-9e54f937-308e-49ce-ab0d-cf70a7b9a8fd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:20.863564 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=58 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:20.869313 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9e54f937-308e-49ce-ab0d-cf70a7b9a8fd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:20.869810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9e54f937-308e-49ce-ab0d-cf70a7b9a8fd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:20.870248 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9e54f937-308e-49ce-ab0d-cf70a7b9a8fd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:20.870968 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9e54f937-308e-49ce-ab0d-cf70a7b9a8fd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:20.871417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9e54f937-308e-49ce-ab0d-cf70a7b9a8fd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:20.871885 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9e54f937-308e-49ce-ab0d-cf70a7b9a8fd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:21.210257 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-9e54f937-308e-49ce-ab0d-cf70a7b9a8fd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "POST /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.355647 Oct 28 09:47:21.217284 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 178/337] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:47:20 2022] POST /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/os-volume_attachments => generated 194 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:47:21.231853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8d0faabe-d38a-41f2-ab46-810d8bbc2be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:21.233024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8d0faabe-d38a-41f2-ab46-810d8bbc2be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:21.241853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d0faabe-d38a-41f2-ab46-810d8bbc2be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:21.242339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d0faabe-d38a-41f2-ab46-810d8bbc2be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:21.242802 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d0faabe-d38a-41f2-ab46-810d8bbc2be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:21.243498 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d0faabe-d38a-41f2-ab46-810d8bbc2be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:21.243946 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d0faabe-d38a-41f2-ab46-810d8bbc2be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:21.244369 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d0faabe-d38a-41f2-ab46-810d8bbc2be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:21.393675 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-16404657", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "networks": [{"uuid": "e6e285c9-7f9a-43e5-8517-efe4e78507ee"}]}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:21.410596 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8d0faabe-d38a-41f2-ab46-810d8bbc2be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:21.420336 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8d0faabe-d38a-41f2-ab46-810d8bbc2be1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1503 microversion: 2.1 time: 0.191536 Oct 28 09:47:21.423996 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 160/338] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:21 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1503 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:21.428296 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7fe3d754-8157-4ee5-a899-463b901fc9e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:21.429308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7fe3d754-8157-4ee5-a899-463b901fc9e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:21.436490 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7fe3d754-8157-4ee5-a899-463b901fc9e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:21.436966 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7fe3d754-8157-4ee5-a899-463b901fc9e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:21.438339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7fe3d754-8157-4ee5-a899-463b901fc9e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:21.439714 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7fe3d754-8157-4ee5-a899-463b901fc9e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:21.440550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7fe3d754-8157-4ee5-a899-463b901fc9e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:21.440988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7fe3d754-8157-4ee5-a899-463b901fc9e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:21.474634 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.network.neutron [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] validate_networks() for [('e6e285c9-7f9a-43e5-8517-efe4e78507ee', None, None, None, None, None)] {{(pid=127029) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:47:21.672048 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7fe3d754-8157-4ee5-a899-463b901fc9e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:21.682552 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7fe3d754-8157-4ee5-a899-463b901fc9e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:21.684086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7fe3d754-8157-4ee5-a899-463b901fc9e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:21.685484 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7fe3d754-8157-4ee5-a899-463b901fc9e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.263265 Oct 28 09:47:21.686145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 161/339] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:21 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:21.690944 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0c29ce13-ab29-49c3-b5a9-30dba3ea931e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ca14c8cf-51d8-4ca4-ae3e-27e7da667346"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:21.695221 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0c29ce13-ab29-49c3-b5a9-30dba3ea931e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:21.701175 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0c29ce13-ab29-49c3-b5a9-30dba3ea931e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:21.701798 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0c29ce13-ab29-49c3-b5a9-30dba3ea931e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:21.702242 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0c29ce13-ab29-49c3-b5a9-30dba3ea931e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:21.703026 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0c29ce13-ab29-49c3-b5a9-30dba3ea931e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:21.703482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0c29ce13-ab29-49c3-b5a9-30dba3ea931e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:21.703995 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0c29ce13-ab29-49c3-b5a9-30dba3ea931e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:21.737731 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-0c29ce13-ab29-49c3-b5a9-30dba3ea931e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] HTTP exception thrown: Cannot 'attach_volume' instance 3b7d7553-45b6-4a10-868e-b653d211245d while it is in vm_state rescued Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0c29ce13-ab29-49c3-b5a9-30dba3ea931e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Request method failure captured: Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: POST /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d/os-volume_attachments HTTP/1.1 Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 74 Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sjq718s-6y9fBFjvzR8wDyF_4AlWYEmdP4icnuJ0s_5turdxYbjJNH08Q4u87xp47MpEwc_KXLuW4AwBVGzdBvRRVA1GVRVASenhMGF1IGhESh28OTGp6LsG6ZaKxtUJblyS6zcH0d96QlXz7dkrheK75c4DcQ-AxWdIWnFfDpBXE Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 227ac6c4e642467ca3b9e0bf7181d5c9 Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:47:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:47:21.741748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/227ac6c4e642467ca3b9e0bf7181d5c9"}]}] Oct 28 09:47:21.741748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:47:21.741748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 227ac6c4e642467ca3b9e0bf7181d5c9 Oct 28 09:47:21.741748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:47:21.741748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-ServerRescueNegativeTestJSON-1296287768-project Oct 28 09:47:21.741748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:47:21.741748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:47:21.741748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: ed9a4c24cbfb4689b7f984219c7503a4 Oct 28 09:47:21.741748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1296287768-project Oct 28 09:47:21.741748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Oct 28 09:47:21.741748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {"volumeAttachment": {"volumeId": "ca14c8cf-51d8-4ca4-ae3e-27e7da667346"}} Oct 28 09:47:21.741748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Cannot 'attach_volume' instance 3b7d7553-45b6-4a10-868e-b653d211245d while it is in vm_state rescued Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'server_id': '3b7d7553-45b6-4a10-868e-b653d211245d', 'body': {'volumeAttachment': {'volumeId': 'ca14c8cf-51d8-4ca4-ae3e-27e7da667346'}}} Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3b7d7553-45b6-4a10-868e-b653d211245d in vm_state rescued. Cannot attach_volume while the instance is in this state. Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:47:21.742636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 3b7d7553-45b6-4a10-868e-b653d211245d while it is in vm_state rescued Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:47:21.743882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:47:21.745095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:47:21.745095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:47:21.745095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 3b7d7553-45b6-4a10-868e-b653d211245d while it is in vm_state rescued Oct 28 09:47:21.745095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:47:21.745095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0c29ce13-ab29-49c3-b5a9-30dba3ea931e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Returning 409 to user: Cannot 'attach_volume' instance 3b7d7553-45b6-4a10-868e-b653d211245d while it is in vm_state rescued {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:47:21.745095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0c29ce13-ab29-49c3-b5a9-30dba3ea931e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "POST /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.054325 Oct 28 09:47:21.746269 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 162/340] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:47:21 2022] POST /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d/os-volume_attachments => generated 152 bytes in 59 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 28 09:47:21.750934 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d730cccf-6cff-4fa6-b8fb-1d7e3a5f44c7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:21.751424 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d730cccf-6cff-4fa6-b8fb-1d7e3a5f44c7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:21.759821 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d730cccf-6cff-4fa6-b8fb-1d7e3a5f44c7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:21.760302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d730cccf-6cff-4fa6-b8fb-1d7e3a5f44c7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:21.760725 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d730cccf-6cff-4fa6-b8fb-1d7e3a5f44c7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:21.761421 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d730cccf-6cff-4fa6-b8fb-1d7e3a5f44c7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:21.762028 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d730cccf-6cff-4fa6-b8fb-1d7e3a5f44c7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:21.762475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d730cccf-6cff-4fa6-b8fb-1d7e3a5f44c7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:21.839088 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d730cccf-6cff-4fa6-b8fb-1d7e3a5f44c7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "POST /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d/action" status: 202 len: 0 microversion: 2.1 time: 0.091536 Oct 28 09:47:21.842882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 163/341] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:47:21 2022] POST /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d/action => generated 0 bytes in 93 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:47:21.849692 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-fd834631-99f7-4f23-b297-963a907e5821 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:21.850559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-fd834631-99f7-4f23-b297-963a907e5821 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:21.858171 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd834631-99f7-4f23-b297-963a907e5821 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:21.858711 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd834631-99f7-4f23-b297-963a907e5821 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:21.859651 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd834631-99f7-4f23-b297-963a907e5821 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:21.860411 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd834631-99f7-4f23-b297-963a907e5821 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:21.860862 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd834631-99f7-4f23-b297-963a907e5821 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:21.861290 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd834631-99f7-4f23-b297-963a907e5821 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:21.889241 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:22.039730 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Flavor limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:47:22.040218 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Image limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:47:22.041133 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Flavor pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:47:22.041643 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Image pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:47:22.042488 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:47:22.054182 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:22.054812 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:22.055381 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:22.056199 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:22.056650 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:22.057281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:22.080419 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Getting quotas for project 9a20b51b10ce463da8f9ad280063bb09. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:47:22.085147 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Getting quotas for user ec23c6593e7a446298d607294997664c and project 9a20b51b10ce463da8f9ad280063bb09. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:47:22.096731 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Going to run 1 instances... {{(pid=127029) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:47:22.099411 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127029) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:47:22.149849 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-fd834631-99f7-4f23-b297-963a907e5821 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:22.161107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-fd834631-99f7-4f23-b297-963a907e5821 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1511 microversion: 2.1 time: 0.316243 Oct 28 09:47:22.162143 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 164/342] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:21 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1511 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:22.164610 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-43f78ce4-0079-47e9-9862-53580f764d29 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.776101 Oct 28 09:47:22.165731 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 179/343] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:47:21 2022] POST /compute/v2.1/servers => generated 380 bytes in 778 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 28 09:47:22.177381 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8c918fb7-7649-4fff-b1ec-e0ceb2d16784 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:22.178237 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8c918fb7-7649-4fff-b1ec-e0ceb2d16784 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:22.185422 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=62 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:22.244814 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8c918fb7-7649-4fff-b1ec-e0ceb2d16784 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:22.257180 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8c918fb7-7649-4fff-b1ec-e0ceb2d16784 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1217 microversion: 2.1 time: 0.084152 Oct 28 09:47:22.258903 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 165/344] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:22 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1217 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:22.546837 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d16d0da8-8be2-45c3-a625-2cb92b76fa4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:22.548278 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d16d0da8-8be2-45c3-a625-2cb92b76fa4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:22.565309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d16d0da8-8be2-45c3-a625-2cb92b76fa4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:22.566085 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d16d0da8-8be2-45c3-a625-2cb92b76fa4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:22.567284 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d16d0da8-8be2-45c3-a625-2cb92b76fa4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:22.568708 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d16d0da8-8be2-45c3-a625-2cb92b76fa4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:22.569339 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d16d0da8-8be2-45c3-a625-2cb92b76fa4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:22.570014 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d16d0da8-8be2-45c3-a625-2cb92b76fa4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:22.608968 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d16d0da8-8be2-45c3-a625-2cb92b76fa4c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/ips" status: 200 len: 151 microversion: 2.1 time: 0.068599 Oct 28 09:47:22.610813 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 180/345] 10.222.0.25 () {58 vars in 1248 bytes} [Fri Oct 28 09:47:22 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/ips => generated 151 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:22.717322 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-32ea7d6d-e07e-4abc-94cb-ed61be9f4dd9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:22.721367 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-32ea7d6d-e07e-4abc-94cb-ed61be9f4dd9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:22.735925 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-32ea7d6d-e07e-4abc-94cb-ed61be9f4dd9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:22.736833 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-32ea7d6d-e07e-4abc-94cb-ed61be9f4dd9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:22.737746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-32ea7d6d-e07e-4abc-94cb-ed61be9f4dd9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:22.738805 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-32ea7d6d-e07e-4abc-94cb-ed61be9f4dd9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:22.739532 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-32ea7d6d-e07e-4abc-94cb-ed61be9f4dd9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:22.740218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-32ea7d6d-e07e-4abc-94cb-ed61be9f4dd9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:22.986901 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-32ea7d6d-e07e-4abc-94cb-ed61be9f4dd9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:23.005643 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-32ea7d6d-e07e-4abc-94cb-ed61be9f4dd9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:23.007768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-32ea7d6d-e07e-4abc-94cb-ed61be9f4dd9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:23.008978 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-32ea7d6d-e07e-4abc-94cb-ed61be9f4dd9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.297746 Oct 28 09:47:23.009868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 166/346] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:22 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:23.189752 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-f007e5ec-91d9-4cd2-8b1f-f0c74ba48895 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:23.190614 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-f007e5ec-91d9-4cd2-8b1f-f0c74ba48895 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:23.199042 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f007e5ec-91d9-4cd2-8b1f-f0c74ba48895 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:23.199558 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f007e5ec-91d9-4cd2-8b1f-f0c74ba48895 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:23.200082 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f007e5ec-91d9-4cd2-8b1f-f0c74ba48895 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:23.200814 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f007e5ec-91d9-4cd2-8b1f-f0c74ba48895 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:23.201338 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f007e5ec-91d9-4cd2-8b1f-f0c74ba48895 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:23.201759 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f007e5ec-91d9-4cd2-8b1f-f0c74ba48895 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:23.285982 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0b92940c-0c1e-4e55-8dc6-797cded0f895 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:23.286840 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0b92940c-0c1e-4e55-8dc6-797cded0f895 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:23.294975 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0b92940c-0c1e-4e55-8dc6-797cded0f895 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:23.295471 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0b92940c-0c1e-4e55-8dc6-797cded0f895 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:23.295906 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0b92940c-0c1e-4e55-8dc6-797cded0f895 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:23.296599 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0b92940c-0c1e-4e55-8dc6-797cded0f895 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:23.297042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0b92940c-0c1e-4e55-8dc6-797cded0f895 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:23.303250 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0b92940c-0c1e-4e55-8dc6-797cded0f895 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:23.360763 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-f007e5ec-91d9-4cd2-8b1f-f0c74ba48895 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:23.369890 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-f007e5ec-91d9-4cd2-8b1f-f0c74ba48895 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1511 microversion: 2.1 time: 0.185115 Oct 28 09:47:23.382768 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 181/347] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:23 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1511 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:23.401095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0b92940c-0c1e-4e55-8dc6-797cded0f895 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:23.412718 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0b92940c-0c1e-4e55-8dc6-797cded0f895 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1269 microversion: 2.1 time: 0.127817 Oct 28 09:47:23.412718 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 167/348] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:23 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1269 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:24.037194 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6db87634-37f7-4db2-b40c-326bb3130a23 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:24.037739 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6db87634-37f7-4db2-b40c-326bb3130a23 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:24.044499 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6db87634-37f7-4db2-b40c-326bb3130a23 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:24.044978 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6db87634-37f7-4db2-b40c-326bb3130a23 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:24.045401 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6db87634-37f7-4db2-b40c-326bb3130a23 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:24.046076 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6db87634-37f7-4db2-b40c-326bb3130a23 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:24.046542 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6db87634-37f7-4db2-b40c-326bb3130a23 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:24.046953 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6db87634-37f7-4db2-b40c-326bb3130a23 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:24.192268 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6db87634-37f7-4db2-b40c-326bb3130a23 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:24.202176 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6db87634-37f7-4db2-b40c-326bb3130a23 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:24.203633 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6db87634-37f7-4db2-b40c-326bb3130a23 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:24.204882 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6db87634-37f7-4db2-b40c-326bb3130a23 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.172149 Oct 28 09:47:24.209723 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 182/349] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:24 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:47:24.227035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6904d249-f785-4848-a9a9-70eeded377fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d2d16afe-439c-43aa-9b38-91658d3478cd"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:24.229777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6904d249-f785-4848-a9a9-70eeded377fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:24.236341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6904d249-f785-4848-a9a9-70eeded377fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:24.236813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6904d249-f785-4848-a9a9-70eeded377fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:24.237242 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6904d249-f785-4848-a9a9-70eeded377fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:24.238023 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6904d249-f785-4848-a9a9-70eeded377fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:24.238500 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6904d249-f785-4848-a9a9-70eeded377fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:24.238919 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6904d249-f785-4848-a9a9-70eeded377fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:24.394929 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-01878eec-cafe-4a04-aa7e-3e0cee215483 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:24.395549 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-01878eec-cafe-4a04-aa7e-3e0cee215483 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:24.402282 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-01878eec-cafe-4a04-aa7e-3e0cee215483 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:24.402809 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-01878eec-cafe-4a04-aa7e-3e0cee215483 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:24.403242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-01878eec-cafe-4a04-aa7e-3e0cee215483 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:24.403941 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-01878eec-cafe-4a04-aa7e-3e0cee215483 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:24.404384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-01878eec-cafe-4a04-aa7e-3e0cee215483 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:24.404799 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-01878eec-cafe-4a04-aa7e-3e0cee215483 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:24.539660 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-01878eec-cafe-4a04-aa7e-3e0cee215483 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:24.547897 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-01878eec-cafe-4a04-aa7e-3e0cee215483 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1511 microversion: 2.1 time: 0.157264 Oct 28 09:47:24.548392 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 183/350] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:24 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1511 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:24.553466 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-570354c4-4d7b-44d1-9d9f-05a850c7e204 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:24.554191 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-570354c4-4d7b-44d1-9d9f-05a850c7e204 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:24.560278 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-570354c4-4d7b-44d1-9d9f-05a850c7e204 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:24.561048 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-570354c4-4d7b-44d1-9d9f-05a850c7e204 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:24.561519 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-570354c4-4d7b-44d1-9d9f-05a850c7e204 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:24.562317 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-570354c4-4d7b-44d1-9d9f-05a850c7e204 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:24.562904 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-570354c4-4d7b-44d1-9d9f-05a850c7e204 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:24.563327 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-570354c4-4d7b-44d1-9d9f-05a850c7e204 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:24.627563 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-570354c4-4d7b-44d1-9d9f-05a850c7e204 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:24.635959 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-570354c4-4d7b-44d1-9d9f-05a850c7e204 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1269 microversion: 2.1 time: 0.086580 Oct 28 09:47:24.636574 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 184/351] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:24 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1269 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:25.197757 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6904d249-f785-4848-a9a9-70eeded377fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "POST /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.974492 Oct 28 09:47:25.204132 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 168/352] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:47:24 2022] POST /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments => generated 194 bytes in 976 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:25.232815 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-58c0c46c-913b-4647-b1b3-af380ff3f7d9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:25.233811 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-58c0c46c-913b-4647-b1b3-af380ff3f7d9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:25.241723 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-58c0c46c-913b-4647-b1b3-af380ff3f7d9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:25.242283 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-58c0c46c-913b-4647-b1b3-af380ff3f7d9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:25.242813 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-58c0c46c-913b-4647-b1b3-af380ff3f7d9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:25.243658 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-58c0c46c-913b-4647-b1b3-af380ff3f7d9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:25.244178 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-58c0c46c-913b-4647-b1b3-af380ff3f7d9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:25.244575 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-58c0c46c-913b-4647-b1b3-af380ff3f7d9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:25.380857 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-58c0c46c-913b-4647-b1b3-af380ff3f7d9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:25.389105 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-58c0c46c-913b-4647-b1b3-af380ff3f7d9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:25.390800 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-58c0c46c-913b-4647-b1b3-af380ff3f7d9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:25.391968 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-58c0c46c-913b-4647-b1b3-af380ff3f7d9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.165304 Oct 28 09:47:25.392543 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 185/353] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:25 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:25.572908 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-337d36f7-ae90-4059-b25a-0d2c751dbede tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:25.573604 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-337d36f7-ae90-4059-b25a-0d2c751dbede tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:25.580374 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-337d36f7-ae90-4059-b25a-0d2c751dbede tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:25.581322 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-337d36f7-ae90-4059-b25a-0d2c751dbede tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:25.582608 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-337d36f7-ae90-4059-b25a-0d2c751dbede tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:25.583335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-337d36f7-ae90-4059-b25a-0d2c751dbede tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:25.607490 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-337d36f7-ae90-4059-b25a-0d2c751dbede tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:25.607490 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-337d36f7-ae90-4059-b25a-0d2c751dbede tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:25.660646 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2eae1b02-d262-412d-b6a9-32e37429721c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:25.661345 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2eae1b02-d262-412d-b6a9-32e37429721c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:25.667985 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2eae1b02-d262-412d-b6a9-32e37429721c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:25.668517 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2eae1b02-d262-412d-b6a9-32e37429721c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:25.668984 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2eae1b02-d262-412d-b6a9-32e37429721c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:25.669693 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2eae1b02-d262-412d-b6a9-32e37429721c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:25.670321 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2eae1b02-d262-412d-b6a9-32e37429721c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:25.670974 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2eae1b02-d262-412d-b6a9-32e37429721c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:25.750575 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2eae1b02-d262-412d-b6a9-32e37429721c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:25.760092 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2eae1b02-d262-412d-b6a9-32e37429721c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1269 microversion: 2.1 time: 0.103655 Oct 28 09:47:25.760672 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 186/354] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:25 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1269 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:25.787299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-337d36f7-ae90-4059-b25a-0d2c751dbede tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:25.795861 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-337d36f7-ae90-4059-b25a-0d2c751dbede tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1517 microversion: 2.1 time: 0.227309 Oct 28 09:47:25.796429 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 169/355] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:25 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1517 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:26.417720 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d9cc82db-3361-47fd-b927-454c3ca069f3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:26.418468 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d9cc82db-3361-47fd-b927-454c3ca069f3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:26.427605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d9cc82db-3361-47fd-b927-454c3ca069f3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:26.428264 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d9cc82db-3361-47fd-b927-454c3ca069f3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:26.428983 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d9cc82db-3361-47fd-b927-454c3ca069f3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:26.429781 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d9cc82db-3361-47fd-b927-454c3ca069f3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:26.430370 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d9cc82db-3361-47fd-b927-454c3ca069f3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:26.430942 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d9cc82db-3361-47fd-b927-454c3ca069f3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:26.633416 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d9cc82db-3361-47fd-b927-454c3ca069f3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:26.642439 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d9cc82db-3361-47fd-b927-454c3ca069f3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:26.643898 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d9cc82db-3361-47fd-b927-454c3ca069f3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:26.645261 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d9cc82db-3361-47fd-b927-454c3ca069f3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.231837 Oct 28 09:47:26.645821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 187/356] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:26 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:26.783470 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6de4f50a-fa2f-4691-b016-028437207bdc tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:26.784150 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6de4f50a-fa2f-4691-b016-028437207bdc tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:26.791390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6de4f50a-fa2f-4691-b016-028437207bdc tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:26.791878 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6de4f50a-fa2f-4691-b016-028437207bdc tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:26.792551 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6de4f50a-fa2f-4691-b016-028437207bdc tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:26.793270 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6de4f50a-fa2f-4691-b016-028437207bdc tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:26.793865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6de4f50a-fa2f-4691-b016-028437207bdc tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:26.794421 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6de4f50a-fa2f-4691-b016-028437207bdc tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:27.013372 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6de4f50a-fa2f-4691-b016-028437207bdc tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:27.021208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6de4f50a-fa2f-4691-b016-028437207bdc tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1317 microversion: 2.1 time: 0.241757 Oct 28 09:47:27.021770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 170/357] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:26 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1317 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:27.232938 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-cddd9bd8-5b97-4701-adab-e21c2e97a477 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Action: 'create', calling method: .version_select at 0x7f271a6035e0>, body: {"keypair": {"name": "tempest-keypair-757421500", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBKMliXX1z7szO3C3bAyQtfUGQ4XhVayXHYrSs0qDquRf85x2GcaGk5TJdiJo7qYDdfDod121ECzjHhA4fS24BeI9OZPqOumbWCkqXl/09UbiheiVPwwVFQe20igTkzyvQ=="}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:27.251076 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-cddd9bd8-5b97-4701-adab-e21c2e97a477 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Getting quotas for project 227ac6c4e642467ca3b9e0bf7181d5c9. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:47:27.255482 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-cddd9bd8-5b97-4701-adab-e21c2e97a477 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Getting quotas for user ed9a4c24cbfb4689b7f984219c7503a4 and project 227ac6c4e642467ca3b9e0bf7181d5c9. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:47:27.279508 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-cddd9bd8-5b97-4701-adab-e21c2e97a477 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.050333 Oct 28 09:47:27.280401 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 188/358] 10.222.0.25 () {60 vars in 1159 bytes} [Fri Oct 28 09:47:27 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:27.671095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-be6bb0af-3cb8-4a83-943f-ba6745a669fb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:27.671691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-be6bb0af-3cb8-4a83-943f-ba6745a669fb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:27.678750 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-be6bb0af-3cb8-4a83-943f-ba6745a669fb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:27.679160 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-be6bb0af-3cb8-4a83-943f-ba6745a669fb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:27.679587 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-be6bb0af-3cb8-4a83-943f-ba6745a669fb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:27.680270 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-be6bb0af-3cb8-4a83-943f-ba6745a669fb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:27.680711 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-be6bb0af-3cb8-4a83-943f-ba6745a669fb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:27.681124 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-be6bb0af-3cb8-4a83-943f-ba6745a669fb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:27.957000 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-be6bb0af-3cb8-4a83-943f-ba6745a669fb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:27.965476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-be6bb0af-3cb8-4a83-943f-ba6745a669fb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:27.967001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-be6bb0af-3cb8-4a83-943f-ba6745a669fb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:27.968169 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-be6bb0af-3cb8-4a83-943f-ba6745a669fb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.301579 Oct 28 09:47:27.968720 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 171/359] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:27 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:28.047610 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-05e03269-ad7a-42ee-bc07-41a3cb817ac3 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:28.049367 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-05e03269-ad7a-42ee-bc07-41a3cb817ac3 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:28.056636 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-05e03269-ad7a-42ee-bc07-41a3cb817ac3 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:28.057196 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-05e03269-ad7a-42ee-bc07-41a3cb817ac3 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:28.058073 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-05e03269-ad7a-42ee-bc07-41a3cb817ac3 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:28.058885 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-05e03269-ad7a-42ee-bc07-41a3cb817ac3 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:28.059504 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-05e03269-ad7a-42ee-bc07-41a3cb817ac3 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:28.059922 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-05e03269-ad7a-42ee-bc07-41a3cb817ac3 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:28.298759 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-09ebaebe-48bd-4b97-b7e3-09e736791b13 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:28.299407 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-09ebaebe-48bd-4b97-b7e3-09e736791b13 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:28.306541 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-09ebaebe-48bd-4b97-b7e3-09e736791b13 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:28.307861 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-09ebaebe-48bd-4b97-b7e3-09e736791b13 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:28.308386 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-09ebaebe-48bd-4b97-b7e3-09e736791b13 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:28.309158 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-09ebaebe-48bd-4b97-b7e3-09e736791b13 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:28.309674 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-09ebaebe-48bd-4b97-b7e3-09e736791b13 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:28.310162 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-09ebaebe-48bd-4b97-b7e3-09e736791b13 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:28.419297 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-05e03269-ad7a-42ee-bc07-41a3cb817ac3 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:28.426343 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-05e03269-ad7a-42ee-bc07-41a3cb817ac3 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1317 microversion: 2.1 time: 0.383703 Oct 28 09:47:28.439122 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 189/360] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:28 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1317 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:28.642199 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-09ebaebe-48bd-4b97-b7e3-09e736791b13 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:28.651512 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-09ebaebe-48bd-4b97-b7e3-09e736791b13 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1594 microversion: 2.1 time: 0.356777 Oct 28 09:47:28.652621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 172/361] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:28 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1594 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:28.708369 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-091944f1-cb8e-4dd4-a8cc-1257306e7ad4 req-75dbdd84-5945-440a-949a-7f5254404c88 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a50a18b7-b11e-48cf-b8dc-58d39afc4d27", "tag": "1c436b7e-fe82-419c-9976-f87b1fa4439c"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:28.721975 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-091944f1-cb8e-4dd4-a8cc-1257306e7ad4 req-75dbdd84-5945-440a-949a-7f5254404c88 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:28.722551 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-091944f1-cb8e-4dd4-a8cc-1257306e7ad4 req-75dbdd84-5945-440a-949a-7f5254404c88 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:28.723179 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-091944f1-cb8e-4dd4-a8cc-1257306e7ad4 req-75dbdd84-5945-440a-949a-7f5254404c88 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:28.730038 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:28.741241 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-091944f1-cb8e-4dd4-a8cc-1257306e7ad4 req-75dbdd84-5945-440a-949a-7f5254404c88 service nova] Creating event network-changed:1c436b7e-fe82-419c-9976-f87b1fa4439c for instance a50a18b7-b11e-48cf-b8dc-58d39afc4d27 on hv-cinder-zed-3 Oct 28 09:47:28.759119 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-091944f1-cb8e-4dd4-a8cc-1257306e7ad4 req-75dbdd84-5945-440a-949a-7f5254404c88 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059982 Oct 28 09:47:28.760514 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 190/362] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:47:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:28.997681 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-18f5028c-2782-40a3-a6c1-060d67366494 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:28.998513 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-18f5028c-2782-40a3-a6c1-060d67366494 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:29.006058 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-18f5028c-2782-40a3-a6c1-060d67366494 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:29.006582 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-18f5028c-2782-40a3-a6c1-060d67366494 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:29.007078 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-18f5028c-2782-40a3-a6c1-060d67366494 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:29.008116 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-18f5028c-2782-40a3-a6c1-060d67366494 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:29.008715 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-18f5028c-2782-40a3-a6c1-060d67366494 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:29.009276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-18f5028c-2782-40a3-a6c1-060d67366494 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:29.271402 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-18f5028c-2782-40a3-a6c1-060d67366494 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:29.283807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-18f5028c-2782-40a3-a6c1-060d67366494 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:29.286080 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-18f5028c-2782-40a3-a6c1-060d67366494 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:29.287679 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-18f5028c-2782-40a3-a6c1-060d67366494 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.295077 Oct 28 09:47:29.288719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 173/363] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:28 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:29.451777 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-7031bd8e-cdb5-49a6-a1f5-ac05495216da tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:29.452616 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-7031bd8e-cdb5-49a6-a1f5-ac05495216da tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:29.461224 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7031bd8e-cdb5-49a6-a1f5-ac05495216da tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:29.461887 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7031bd8e-cdb5-49a6-a1f5-ac05495216da tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:29.462649 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7031bd8e-cdb5-49a6-a1f5-ac05495216da tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:29.463504 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7031bd8e-cdb5-49a6-a1f5-ac05495216da tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:29.464104 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7031bd8e-cdb5-49a6-a1f5-ac05495216da tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:29.464827 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7031bd8e-cdb5-49a6-a1f5-ac05495216da tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:29.707412 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7031bd8e-cdb5-49a6-a1f5-ac05495216da tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:29.715667 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-7031bd8e-cdb5-49a6-a1f5-ac05495216da tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1317 microversion: 2.1 time: 0.268057 Oct 28 09:47:29.730872 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 191/364] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:29 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1317 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:29.920485 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2146105579", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "51c62d53-c6c7-4bb9-b1b6-d47428b64a8c"}], "security_groups": [{"name": "tempest-securitygroup--1226955048"}], "key_name": "tempest-keypair-757421500", "user_data": "}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:30.255360 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.network.neutron [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] validate_networks() for [('51c62d53-c6c7-4bb9-b1b6-d47428b64a8c', None, None, None, None, None)] {{(pid=127030) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:47:30.313390 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-f116d62c-117a-4bc7-aac9-ae9e997e0eee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:30.314061 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-f116d62c-117a-4bc7-aac9-ae9e997e0eee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:30.321416 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f116d62c-117a-4bc7-aac9-ae9e997e0eee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:30.321904 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f116d62c-117a-4bc7-aac9-ae9e997e0eee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:30.322341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f116d62c-117a-4bc7-aac9-ae9e997e0eee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:30.324235 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f116d62c-117a-4bc7-aac9-ae9e997e0eee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:30.324700 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f116d62c-117a-4bc7-aac9-ae9e997e0eee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:30.325119 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f116d62c-117a-4bc7-aac9-ae9e997e0eee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:30.562781 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-f116d62c-117a-4bc7-aac9-ae9e997e0eee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:30.576941 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-f116d62c-117a-4bc7-aac9-ae9e997e0eee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:30.578313 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-f116d62c-117a-4bc7-aac9-ae9e997e0eee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:30.579927 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-f116d62c-117a-4bc7-aac9-ae9e997e0eee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.270297 Oct 28 09:47:30.581209 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 192/365] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:30 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:47:30.740371 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b927aea5-2e3e-40cc-81f5-e4fea5d99349 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:30.741279 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b927aea5-2e3e-40cc-81f5-e4fea5d99349 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:30.747493 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b927aea5-2e3e-40cc-81f5-e4fea5d99349 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:30.747983 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b927aea5-2e3e-40cc-81f5-e4fea5d99349 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:30.748613 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b927aea5-2e3e-40cc-81f5-e4fea5d99349 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:30.749463 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b927aea5-2e3e-40cc-81f5-e4fea5d99349 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:30.749916 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b927aea5-2e3e-40cc-81f5-e4fea5d99349 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:30.750520 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b927aea5-2e3e-40cc-81f5-e4fea5d99349 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:30.822028 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Flavor limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:47:30.822663 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Image limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:47:30.823490 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Flavor pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:47:30.823973 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Image pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:47:30.825067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:47:30.835040 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:30.835669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:30.836207 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:30.837043 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:30.837611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:30.838133 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:30.856619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Getting quotas for project 227ac6c4e642467ca3b9e0bf7181d5c9. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:47:30.863277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Getting quotas for user ed9a4c24cbfb4689b7f984219c7503a4 and project 227ac6c4e642467ca3b9e0bf7181d5c9. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:47:30.879504 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Going to run 1 instances... {{(pid=127030) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:47:30.882660 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127030) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:47:30.948355 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-33572101-4f3e-46d7-a12a-a89da384de56 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.032825 Oct 28 09:47:30.958731 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 174/366] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:47:29 2022] POST /compute/v2.1/servers => generated 409 bytes in 1043 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 09:47:30.971036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-3c937fd8-8295-499b-9dbe-f4898000cb1b req-e1301772-bb5f-4cd5-9b2c-0050681221a3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "04eedf41-b3f2-4b33-b524-988daff99fdc", "tag": "0ec06b02-a12d-496d-a40d-c56118810a83"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:30.994841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-3c937fd8-8295-499b-9dbe-f4898000cb1b req-e1301772-bb5f-4cd5-9b2c-0050681221a3 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:30.994841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-3c937fd8-8295-499b-9dbe-f4898000cb1b req-e1301772-bb5f-4cd5-9b2c-0050681221a3 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:30.994841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-3c937fd8-8295-499b-9dbe-f4898000cb1b req-e1301772-bb5f-4cd5-9b2c-0050681221a3 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:31.014922 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-3c937fd8-8295-499b-9dbe-f4898000cb1b req-e1301772-bb5f-4cd5-9b2c-0050681221a3 service nova] Creating event network-changed:0ec06b02-a12d-496d-a40d-c56118810a83 for instance 04eedf41-b3f2-4b33-b524-988daff99fdc on hv-cinder-zed-3 Oct 28 09:47:31.022944 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-3c937fd8-8295-499b-9dbe-f4898000cb1b req-e1301772-bb5f-4cd5-9b2c-0050681221a3 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060680 Oct 28 09:47:31.025155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 175/367] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:47:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:31.039657 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-26e96861-0526-4282-a3b9-35b34fc30806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:31.043118 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-26e96861-0526-4282-a3b9-35b34fc30806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:31.132145 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b927aea5-2e3e-40cc-81f5-e4fea5d99349 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:31.144984 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b927aea5-2e3e-40cc-81f5-e4fea5d99349 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1317 microversion: 2.1 time: 0.408370 Oct 28 09:47:31.145959 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 193/368] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:30 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1317 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:31.233860 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-26e96861-0526-4282-a3b9-35b34fc30806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:31.244038 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-26e96861-0526-4282-a3b9-35b34fc30806 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1245 microversion: 2.1 time: 0.211579 Oct 28 09:47:31.246609 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 176/369] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:31 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1245 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:31.607734 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-ad25b8b6-c70a-4a18-a5c1-f836ab88fc91 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:31.608264 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-ad25b8b6-c70a-4a18-a5c1-f836ab88fc91 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:31.618054 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ad25b8b6-c70a-4a18-a5c1-f836ab88fc91 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:31.620121 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ad25b8b6-c70a-4a18-a5c1-f836ab88fc91 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:31.630347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ad25b8b6-c70a-4a18-a5c1-f836ab88fc91 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:31.636439 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ad25b8b6-c70a-4a18-a5c1-f836ab88fc91 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:31.637994 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ad25b8b6-c70a-4a18-a5c1-f836ab88fc91 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:31.638916 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ad25b8b6-c70a-4a18-a5c1-f836ab88fc91 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:31.998955 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-ad25b8b6-c70a-4a18-a5c1-f836ab88fc91 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:32.020851 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-ad25b8b6-c70a-4a18-a5c1-f836ab88fc91 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:32.032796 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-ad25b8b6-c70a-4a18-a5c1-f836ab88fc91 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:32.035116 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-ad25b8b6-c70a-4a18-a5c1-f836ab88fc91 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.431628 Oct 28 09:47:32.043626 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 194/370] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:31 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:32.171824 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8d9dab14-d3a9-4a61-82ce-c07e48c6c8af tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:32.172435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8d9dab14-d3a9-4a61-82ce-c07e48c6c8af tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:32.185315 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d9dab14-d3a9-4a61-82ce-c07e48c6c8af tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:32.185782 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d9dab14-d3a9-4a61-82ce-c07e48c6c8af tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:32.186238 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d9dab14-d3a9-4a61-82ce-c07e48c6c8af tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:32.187668 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d9dab14-d3a9-4a61-82ce-c07e48c6c8af tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:32.188489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d9dab14-d3a9-4a61-82ce-c07e48c6c8af tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:32.189625 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d9dab14-d3a9-4a61-82ce-c07e48c6c8af tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:32.274711 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6b133e79-ca0c-4456-b301-f61fcf352c26 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:32.275473 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6b133e79-ca0c-4456-b301-f61fcf352c26 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:32.281206 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=4 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:32.283835 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6b133e79-ca0c-4456-b301-f61fcf352c26 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:32.284713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6b133e79-ca0c-4456-b301-f61fcf352c26 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:32.285386 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6b133e79-ca0c-4456-b301-f61fcf352c26 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:32.286413 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6b133e79-ca0c-4456-b301-f61fcf352c26 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:32.287139 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6b133e79-ca0c-4456-b301-f61fcf352c26 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:32.287792 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6b133e79-ca0c-4456-b301-f61fcf352c26 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:32.420650 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6b133e79-ca0c-4456-b301-f61fcf352c26 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:32.428819 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6b133e79-ca0c-4456-b301-f61fcf352c26 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1297 microversion: 2.1 time: 0.164017 Oct 28 09:47:32.429425 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 195/371] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:32 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1297 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:32.490777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8d9dab14-d3a9-4a61-82ce-c07e48c6c8af tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:32.516368 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8d9dab14-d3a9-4a61-82ce-c07e48c6c8af tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1317 microversion: 2.1 time: 0.348606 Oct 28 09:47:32.531712 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 177/372] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:32 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1317 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:32.901778 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-7fa0cc68-0aa8-4c46-bcf3-868c30ee5412 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-907287077"}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:32.902471 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-7fa0cc68-0aa8-4c46-bcf3-868c30ee5412 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:32.912819 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fa0cc68-0aa8-4c46-bcf3-868c30ee5412 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:32.915993 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fa0cc68-0aa8-4c46-bcf3-868c30ee5412 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:32.916642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fa0cc68-0aa8-4c46-bcf3-868c30ee5412 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:32.918781 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fa0cc68-0aa8-4c46-bcf3-868c30ee5412 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:32.919449 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fa0cc68-0aa8-4c46-bcf3-868c30ee5412 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:32.920704 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fa0cc68-0aa8-4c46-bcf3-868c30ee5412 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:33.061686 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-fd37e81c-aacc-41ea-bab7-290fcd00af1d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:33.062383 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-fd37e81c-aacc-41ea-bab7-290fcd00af1d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:33.069994 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd37e81c-aacc-41ea-bab7-290fcd00af1d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:33.074098 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd37e81c-aacc-41ea-bab7-290fcd00af1d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:33.074783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd37e81c-aacc-41ea-bab7-290fcd00af1d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:33.075687 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd37e81c-aacc-41ea-bab7-290fcd00af1d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:33.076296 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd37e81c-aacc-41ea-bab7-290fcd00af1d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:33.076865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd37e81c-aacc-41ea-bab7-290fcd00af1d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:33.211925 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.network.security_group_api [None req-7fa0cc68-0aa8-4c46-bcf3-868c30ee5412 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Adding security group b42bf8e7-036b-494e-937c-9d6eeea54ddc to port 0ec06b02-a12d-496d-a40d-c56118810a83 Oct 28 09:47:33.464485 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-fd37e81c-aacc-41ea-bab7-290fcd00af1d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:33.473532 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-fd37e81c-aacc-41ea-bab7-290fcd00af1d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:33.475092 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-fd37e81c-aacc-41ea-bab7-290fcd00af1d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:33.476396 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-fd37e81c-aacc-41ea-bab7-290fcd00af1d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.418586 Oct 28 09:47:33.486440 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 178/373] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:33 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:33.486440 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8f6d834d-e94e-4544-ab36-72e807c267dc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:33.486440 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8f6d834d-e94e-4544-ab36-72e807c267dc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:33.490009 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8f6d834d-e94e-4544-ab36-72e807c267dc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:33.490641 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8f6d834d-e94e-4544-ab36-72e807c267dc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:33.491256 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8f6d834d-e94e-4544-ab36-72e807c267dc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:33.492201 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8f6d834d-e94e-4544-ab36-72e807c267dc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:33.492904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8f6d834d-e94e-4544-ab36-72e807c267dc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:33.493426 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8f6d834d-e94e-4544-ab36-72e807c267dc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:33.584132 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8f6d834d-e94e-4544-ab36-72e807c267dc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:33.595223 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8f6d834d-e94e-4544-ab36-72e807c267dc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1303 microversion: 2.1 time: 0.116440 Oct 28 09:47:33.608520 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 179/374] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:33 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1303 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:33.615637 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-377f01e5-3b9d-4d28-b469-fcc6d1fd4aa8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:33.617851 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-377f01e5-3b9d-4d28-b469-fcc6d1fd4aa8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:33.624673 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-377f01e5-3b9d-4d28-b469-fcc6d1fd4aa8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:33.625457 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-377f01e5-3b9d-4d28-b469-fcc6d1fd4aa8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:33.626080 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-377f01e5-3b9d-4d28-b469-fcc6d1fd4aa8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:33.627168 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-377f01e5-3b9d-4d28-b469-fcc6d1fd4aa8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:33.628189 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-377f01e5-3b9d-4d28-b469-fcc6d1fd4aa8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:33.629113 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-377f01e5-3b9d-4d28-b469-fcc6d1fd4aa8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:34.182272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-377f01e5-3b9d-4d28-b469-fcc6d1fd4aa8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:34.196569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-377f01e5-3b9d-4d28-b469-fcc6d1fd4aa8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1317 microversion: 2.1 time: 0.583746 Oct 28 09:47:34.198243 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 180/375] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:33 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1317 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:34.244143 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-7fa0cc68-0aa8-4c46-bcf3-868c30ee5412 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "POST /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/action" status: 202 len: 0 microversion: 2.1 time: 1.347099 Oct 28 09:47:34.248103 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 196/376] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:47:32 2022] POST /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/action => generated 0 bytes in 1349 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:47:34.256695 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-81195185-2118-4c75-a871-030d04729e23 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:34.257664 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-81195185-2118-4c75-a871-030d04729e23 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:34.264247 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-81195185-2118-4c75-a871-030d04729e23 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:34.264732 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-81195185-2118-4c75-a871-030d04729e23 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:34.265162 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-81195185-2118-4c75-a871-030d04729e23 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:34.265855 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-81195185-2118-4c75-a871-030d04729e23 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:34.266301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-81195185-2118-4c75-a871-030d04729e23 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:34.266744 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-81195185-2118-4c75-a871-030d04729e23 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:34.286218 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-17d164c9-c7f8-4a54-825d-856285bcdbc9 req-eef319c3-e294-4e66-9b87-dbfca3036bd4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "04eedf41-b3f2-4b33-b524-988daff99fdc", "tag": "0ec06b02-a12d-496d-a40d-c56118810a83"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:34.297800 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-17d164c9-c7f8-4a54-825d-856285bcdbc9 req-eef319c3-e294-4e66-9b87-dbfca3036bd4 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:34.299278 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-17d164c9-c7f8-4a54-825d-856285bcdbc9 req-eef319c3-e294-4e66-9b87-dbfca3036bd4 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:34.300834 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-17d164c9-c7f8-4a54-825d-856285bcdbc9 req-eef319c3-e294-4e66-9b87-dbfca3036bd4 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:34.321073 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-17d164c9-c7f8-4a54-825d-856285bcdbc9 req-eef319c3-e294-4e66-9b87-dbfca3036bd4 service nova] Creating event network-changed:0ec06b02-a12d-496d-a40d-c56118810a83 for instance 04eedf41-b3f2-4b33-b524-988daff99fdc on hv-cinder-zed-3 Oct 28 09:47:34.328372 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-17d164c9-c7f8-4a54-825d-856285bcdbc9 req-eef319c3-e294-4e66-9b87-dbfca3036bd4 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046680 Oct 28 09:47:34.329391 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 197/377] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:47:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:34.514263 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-720ff774-b754-4888-8d6f-0377d1bb7973 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:34.515412 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-720ff774-b754-4888-8d6f-0377d1bb7973 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:34.523392 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-720ff774-b754-4888-8d6f-0377d1bb7973 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:34.524138 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-720ff774-b754-4888-8d6f-0377d1bb7973 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:34.524787 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-720ff774-b754-4888-8d6f-0377d1bb7973 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:34.525885 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-720ff774-b754-4888-8d6f-0377d1bb7973 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:34.526676 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-720ff774-b754-4888-8d6f-0377d1bb7973 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:34.527248 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-720ff774-b754-4888-8d6f-0377d1bb7973 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:34.651524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-81195185-2118-4c75-a871-030d04729e23 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:34.662935 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-81195185-2118-4c75-a871-030d04729e23 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1758 microversion: 2.1 time: 0.413181 Oct 28 09:47:34.663591 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 181/378] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:34 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1758 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:34.674313 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c7a23f8a-3620-48d6-9408-59ff5f50dedd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:34.676150 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c7a23f8a-3620-48d6-9408-59ff5f50dedd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:34.692598 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c7a23f8a-3620-48d6-9408-59ff5f50dedd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:34.693087 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c7a23f8a-3620-48d6-9408-59ff5f50dedd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:34.693525 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c7a23f8a-3620-48d6-9408-59ff5f50dedd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:34.694246 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c7a23f8a-3620-48d6-9408-59ff5f50dedd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:34.694691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c7a23f8a-3620-48d6-9408-59ff5f50dedd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:34.695332 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c7a23f8a-3620-48d6-9408-59ff5f50dedd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:35.002491 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-720ff774-b754-4888-8d6f-0377d1bb7973 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:35.023497 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-720ff774-b754-4888-8d6f-0377d1bb7973 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:35.026383 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-720ff774-b754-4888-8d6f-0377d1bb7973 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:35.027745 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-720ff774-b754-4888-8d6f-0377d1bb7973 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.518435 Oct 28 09:47:35.029181 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 198/379] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:34 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:47:35.111562 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c7a23f8a-3620-48d6-9408-59ff5f50dedd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:35.123737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c7a23f8a-3620-48d6-9408-59ff5f50dedd tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1371 microversion: 2.1 time: 0.457701 Oct 28 09:47:35.136923 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 182/380] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:34 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1371 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:35.227610 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-23a5f9c6-15e1-459b-b2c6-1bf6141fca22 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:35.228564 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-23a5f9c6-15e1-459b-b2c6-1bf6141fca22 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:35.235774 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-23a5f9c6-15e1-459b-b2c6-1bf6141fca22 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:35.236695 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-23a5f9c6-15e1-459b-b2c6-1bf6141fca22 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:35.237635 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-23a5f9c6-15e1-459b-b2c6-1bf6141fca22 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:35.239643 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-23a5f9c6-15e1-459b-b2c6-1bf6141fca22 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:35.240348 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-23a5f9c6-15e1-459b-b2c6-1bf6141fca22 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:35.240970 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-23a5f9c6-15e1-459b-b2c6-1bf6141fca22 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:35.449153 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-61aa6337-fc7e-4dfb-a54e-0ca8788b686f req-05517d5c-d92d-4fa4-b16d-309d3dd7ee49 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a50a18b7-b11e-48cf-b8dc-58d39afc4d27", "name": "network-vif-plugged", "status": "completed", "tag": "1c436b7e-fe82-419c-9976-f87b1fa4439c"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:35.468146 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-61aa6337-fc7e-4dfb-a54e-0ca8788b686f req-05517d5c-d92d-4fa4-b16d-309d3dd7ee49 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:35.469269 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-61aa6337-fc7e-4dfb-a54e-0ca8788b686f req-05517d5c-d92d-4fa4-b16d-309d3dd7ee49 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:35.469856 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-61aa6337-fc7e-4dfb-a54e-0ca8788b686f req-05517d5c-d92d-4fa4-b16d-309d3dd7ee49 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:35.474983 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:35.495065 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-61aa6337-fc7e-4dfb-a54e-0ca8788b686f req-05517d5c-d92d-4fa4-b16d-309d3dd7ee49 service nova] Creating event network-vif-plugged:1c436b7e-fe82-419c-9976-f87b1fa4439c for instance a50a18b7-b11e-48cf-b8dc-58d39afc4d27 on hv-cinder-zed-3 Oct 28 09:47:35.504429 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-61aa6337-fc7e-4dfb-a54e-0ca8788b686f req-05517d5c-d92d-4fa4-b16d-309d3dd7ee49 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059500 Oct 28 09:47:35.506152 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 183/381] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:47:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:35.512779 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b366ff2a-614a-4b93-9ceb-7a904000bb2e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:35.523710 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b366ff2a-614a-4b93-9ceb-7a904000bb2e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:35.524194 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b366ff2a-614a-4b93-9ceb-7a904000bb2e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:35.526304 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b366ff2a-614a-4b93-9ceb-7a904000bb2e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:35.600714 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b366ff2a-614a-4b93-9ceb-7a904000bb2e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Going to try to stop instance {{(pid=127030) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 28 09:47:35.614949 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-23a5f9c6-15e1-459b-b2c6-1bf6141fca22 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:35.625362 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-23a5f9c6-15e1-459b-b2c6-1bf6141fca22 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1317 microversion: 2.1 time: 0.400907 Oct 28 09:47:35.640567 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 199/382] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:35 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1317 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:35.669408 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b366ff2a-614a-4b93-9ceb-7a904000bb2e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "POST /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/action" status: 202 len: 0 microversion: 2.1 time: 0.161539 Oct 28 09:47:35.669985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 184/383] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:47:35 2022] POST /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/action => generated 0 bytes in 163 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:47:35.679277 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-9d0925cd-688a-41af-ad21-3e885c3ddd8f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:35.680088 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-9d0925cd-688a-41af-ad21-3e885c3ddd8f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:35.695817 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9d0925cd-688a-41af-ad21-3e885c3ddd8f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:35.698683 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9d0925cd-688a-41af-ad21-3e885c3ddd8f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:35.702121 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9d0925cd-688a-41af-ad21-3e885c3ddd8f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:35.703612 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9d0925cd-688a-41af-ad21-3e885c3ddd8f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:35.704688 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9d0925cd-688a-41af-ad21-3e885c3ddd8f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:35.706241 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9d0925cd-688a-41af-ad21-3e885c3ddd8f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:35.949469 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-9d0925cd-688a-41af-ad21-3e885c3ddd8f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:35.958952 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-9d0925cd-688a-41af-ad21-3e885c3ddd8f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.284099 Oct 28 09:47:35.959530 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 200/384] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:35 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:36.057868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8c5081f0-1334-49ed-8382-c685e7338cc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:36.058704 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8c5081f0-1334-49ed-8382-c685e7338cc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:36.065043 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8c5081f0-1334-49ed-8382-c685e7338cc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:36.065647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8c5081f0-1334-49ed-8382-c685e7338cc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:36.066431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8c5081f0-1334-49ed-8382-c685e7338cc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:36.067222 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8c5081f0-1334-49ed-8382-c685e7338cc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:36.067731 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8c5081f0-1334-49ed-8382-c685e7338cc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:36.069019 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8c5081f0-1334-49ed-8382-c685e7338cc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:36.151210 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5dde2326-f5bd-484a-a6f7-98bf12eb8754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:36.152205 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5dde2326-f5bd-484a-a6f7-98bf12eb8754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:36.167832 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5dde2326-f5bd-484a-a6f7-98bf12eb8754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:36.168418 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5dde2326-f5bd-484a-a6f7-98bf12eb8754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:36.168984 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5dde2326-f5bd-484a-a6f7-98bf12eb8754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:36.170683 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5dde2326-f5bd-484a-a6f7-98bf12eb8754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:36.171408 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5dde2326-f5bd-484a-a6f7-98bf12eb8754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:36.171930 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5dde2326-f5bd-484a-a6f7-98bf12eb8754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:36.404105 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8c5081f0-1334-49ed-8382-c685e7338cc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:36.412395 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8c5081f0-1334-49ed-8382-c685e7338cc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:36.413843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8c5081f0-1334-49ed-8382-c685e7338cc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:36.415034 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8c5081f0-1334-49ed-8382-c685e7338cc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.364817 Oct 28 09:47:36.434342 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 185/385] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:36 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:47:36.467405 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5dde2326-f5bd-484a-a6f7-98bf12eb8754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:36.475955 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5dde2326-f5bd-484a-a6f7-98bf12eb8754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1371 microversion: 2.1 time: 0.328638 Oct 28 09:47:36.476949 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 201/386] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:36 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1371 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:36.531134 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-f7b97782-3bdc-42af-8752-34adfd040bc8 req-31136344-47dd-4b49-acd5-7b1e4dc8796a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4891b63b-eb08-448d-a640-f9579c8140ff", "tag": "6e78f30d-6812-43c6-910d-70f563ee8ad9"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:36.541509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-f7b97782-3bdc-42af-8752-34adfd040bc8 req-31136344-47dd-4b49-acd5-7b1e4dc8796a service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:36.542248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-f7b97782-3bdc-42af-8752-34adfd040bc8 req-31136344-47dd-4b49-acd5-7b1e4dc8796a service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:36.543181 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-f7b97782-3bdc-42af-8752-34adfd040bc8 req-31136344-47dd-4b49-acd5-7b1e4dc8796a service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:36.555666 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-f7b97782-3bdc-42af-8752-34adfd040bc8 req-31136344-47dd-4b49-acd5-7b1e4dc8796a service nova] Creating event network-changed:6e78f30d-6812-43c6-910d-70f563ee8ad9 for instance 4891b63b-eb08-448d-a640-f9579c8140ff on hv-cinder-zed-3 Oct 28 09:47:36.561011 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-f7b97782-3bdc-42af-8752-34adfd040bc8 req-31136344-47dd-4b49-acd5-7b1e4dc8796a service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033458 Oct 28 09:47:36.562114 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 186/387] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:47:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:36.649424 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3701565b-4637-452f-b9ba-2a7931c8aea7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:36.650038 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-3701565b-4637-452f-b9ba-2a7931c8aea7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:36.656656 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3701565b-4637-452f-b9ba-2a7931c8aea7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:36.657173 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3701565b-4637-452f-b9ba-2a7931c8aea7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:36.657603 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3701565b-4637-452f-b9ba-2a7931c8aea7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:36.658289 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3701565b-4637-452f-b9ba-2a7931c8aea7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:36.658972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3701565b-4637-452f-b9ba-2a7931c8aea7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:36.659393 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3701565b-4637-452f-b9ba-2a7931c8aea7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:36.806863 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3701565b-4637-452f-b9ba-2a7931c8aea7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:36.814554 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-3701565b-4637-452f-b9ba-2a7931c8aea7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1317 microversion: 2.1 time: 0.170141 Oct 28 09:47:36.815111 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 202/388] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:36 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1317 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:36.982971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-174b1202-cbc9-40aa-a79b-1bdfbcf931f0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:36.983607 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-174b1202-cbc9-40aa-a79b-1bdfbcf931f0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:36.990280 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-174b1202-cbc9-40aa-a79b-1bdfbcf931f0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:36.990785 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-174b1202-cbc9-40aa-a79b-1bdfbcf931f0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:36.991222 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-174b1202-cbc9-40aa-a79b-1bdfbcf931f0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:36.991902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-174b1202-cbc9-40aa-a79b-1bdfbcf931f0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:36.992351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-174b1202-cbc9-40aa-a79b-1bdfbcf931f0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:36.992771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-174b1202-cbc9-40aa-a79b-1bdfbcf931f0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:37.123192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-174b1202-cbc9-40aa-a79b-1bdfbcf931f0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:37.131564 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-174b1202-cbc9-40aa-a79b-1bdfbcf931f0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.152873 Oct 28 09:47:37.132127 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 187/389] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:36 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:37.442842 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6c3ed20c-6a19-4dd8-b976-922c372fcfaf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:37.443473 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6c3ed20c-6a19-4dd8-b976-922c372fcfaf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:37.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=10 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:37.451261 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6c3ed20c-6a19-4dd8-b976-922c372fcfaf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:37.451807 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6c3ed20c-6a19-4dd8-b976-922c372fcfaf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:37.452243 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6c3ed20c-6a19-4dd8-b976-922c372fcfaf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:37.452943 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6c3ed20c-6a19-4dd8-b976-922c372fcfaf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:37.453389 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6c3ed20c-6a19-4dd8-b976-922c372fcfaf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:37.453895 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6c3ed20c-6a19-4dd8-b976-922c372fcfaf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:37.499337 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6334b67f-08d0-4532-8f20-f6594c7b73be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:37.500891 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6334b67f-08d0-4532-8f20-f6594c7b73be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:37.507172 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6334b67f-08d0-4532-8f20-f6594c7b73be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:37.507743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6334b67f-08d0-4532-8f20-f6594c7b73be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:37.508242 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6334b67f-08d0-4532-8f20-f6594c7b73be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:37.508978 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6334b67f-08d0-4532-8f20-f6594c7b73be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:37.509420 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6334b67f-08d0-4532-8f20-f6594c7b73be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:37.509864 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6334b67f-08d0-4532-8f20-f6594c7b73be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:37.713999 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6c3ed20c-6a19-4dd8-b976-922c372fcfaf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:37.723699 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6c3ed20c-6a19-4dd8-b976-922c372fcfaf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:37.725320 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6c3ed20c-6a19-4dd8-b976-922c372fcfaf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:37.726594 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6c3ed20c-6a19-4dd8-b976-922c372fcfaf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.288300 Oct 28 09:47:37.727266 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 203/390] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:37 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:37.740215 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6334b67f-08d0-4532-8f20-f6594c7b73be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:37.748222 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6334b67f-08d0-4532-8f20-f6594c7b73be tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1371 microversion: 2.1 time: 0.252698 Oct 28 09:47:37.748780 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 188/391] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:37 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1371 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:37.842212 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-612b826d-3cb2-4997-911a-3b9cf69cc1c7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:37.843041 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-612b826d-3cb2-4997-911a-3b9cf69cc1c7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:37.850041 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-612b826d-3cb2-4997-911a-3b9cf69cc1c7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:37.850593 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-612b826d-3cb2-4997-911a-3b9cf69cc1c7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:37.851101 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-612b826d-3cb2-4997-911a-3b9cf69cc1c7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:37.852128 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-612b826d-3cb2-4997-911a-3b9cf69cc1c7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:37.852825 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-612b826d-3cb2-4997-911a-3b9cf69cc1c7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:37.853305 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-612b826d-3cb2-4997-911a-3b9cf69cc1c7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:37.992726 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-612b826d-3cb2-4997-911a-3b9cf69cc1c7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:38.000469 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-612b826d-3cb2-4997-911a-3b9cf69cc1c7 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1317 microversion: 2.1 time: 0.166756 Oct 28 09:47:38.001047 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 204/392] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:37 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1317 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:38.155679 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-feea0f3e-e9a8-4004-8c7c-ede980f170f6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:38.156365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-feea0f3e-e9a8-4004-8c7c-ede980f170f6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:38.162525 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-feea0f3e-e9a8-4004-8c7c-ede980f170f6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:38.163009 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-feea0f3e-e9a8-4004-8c7c-ede980f170f6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:38.163432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-feea0f3e-e9a8-4004-8c7c-ede980f170f6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:38.164114 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-feea0f3e-e9a8-4004-8c7c-ede980f170f6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:38.164569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-feea0f3e-e9a8-4004-8c7c-ede980f170f6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:38.164968 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-feea0f3e-e9a8-4004-8c7c-ede980f170f6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:38.308007 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-feea0f3e-e9a8-4004-8c7c-ede980f170f6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:38.316645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-feea0f3e-e9a8-4004-8c7c-ede980f170f6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.164787 Oct 28 09:47:38.317209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 189/393] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:38 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:38.755829 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3914acb4-f88d-49e7-93a4-5edecbe852e5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:38.756394 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-3914acb4-f88d-49e7-93a4-5edecbe852e5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:38.763154 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3914acb4-f88d-49e7-93a4-5edecbe852e5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:38.763739 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3914acb4-f88d-49e7-93a4-5edecbe852e5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:38.764225 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3914acb4-f88d-49e7-93a4-5edecbe852e5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:38.765036 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3914acb4-f88d-49e7-93a4-5edecbe852e5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:38.765506 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3914acb4-f88d-49e7-93a4-5edecbe852e5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:38.766031 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3914acb4-f88d-49e7-93a4-5edecbe852e5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:38.772231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8f1999f5-fdea-4dcc-b1d5-e6053a3615eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:38.773053 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8f1999f5-fdea-4dcc-b1d5-e6053a3615eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:38.780207 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8f1999f5-fdea-4dcc-b1d5-e6053a3615eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:38.781025 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8f1999f5-fdea-4dcc-b1d5-e6053a3615eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:38.781627 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8f1999f5-fdea-4dcc-b1d5-e6053a3615eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:38.782459 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8f1999f5-fdea-4dcc-b1d5-e6053a3615eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:38.783021 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8f1999f5-fdea-4dcc-b1d5-e6053a3615eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:38.783701 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8f1999f5-fdea-4dcc-b1d5-e6053a3615eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:39.227782 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3914acb4-f88d-49e7-93a4-5edecbe852e5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:39.231886 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8f1999f5-fdea-4dcc-b1d5-e6053a3615eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:39.236327 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3914acb4-f88d-49e7-93a4-5edecbe852e5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:39.237792 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3914acb4-f88d-49e7-93a4-5edecbe852e5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:39.238987 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-3914acb4-f88d-49e7-93a4-5edecbe852e5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.489348 Oct 28 09:47:39.239547 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 205/394] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:38 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:39.240960 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8f1999f5-fdea-4dcc-b1d5-e6053a3615eb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1371 microversion: 2.1 time: 0.473560 Oct 28 09:47:39.241591 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 190/395] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:38 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1371 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:39.243978 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-fedc301d-476d-450e-aa60-84ec94d74bee tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:39.244680 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-fedc301d-476d-450e-aa60-84ec94d74bee tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:39.251093 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fedc301d-476d-450e-aa60-84ec94d74bee tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:39.251624 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fedc301d-476d-450e-aa60-84ec94d74bee tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:39.252316 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fedc301d-476d-450e-aa60-84ec94d74bee tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:39.253029 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fedc301d-476d-450e-aa60-84ec94d74bee tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:39.253973 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fedc301d-476d-450e-aa60-84ec94d74bee tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:39.254373 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fedc301d-476d-450e-aa60-84ec94d74bee tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:39.339811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ff271b74-b876-4005-9fbf-3512d8d40569 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:39.340418 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ff271b74-b876-4005-9fbf-3512d8d40569 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:39.346480 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ff271b74-b876-4005-9fbf-3512d8d40569 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:39.346962 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ff271b74-b876-4005-9fbf-3512d8d40569 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:39.347428 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ff271b74-b876-4005-9fbf-3512d8d40569 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:39.348112 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ff271b74-b876-4005-9fbf-3512d8d40569 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:39.348587 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ff271b74-b876-4005-9fbf-3512d8d40569 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:39.349004 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ff271b74-b876-4005-9fbf-3512d8d40569 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:39.402875 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-fedc301d-476d-450e-aa60-84ec94d74bee tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:39.410869 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-fedc301d-476d-450e-aa60-84ec94d74bee tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1506 microversion: 2.1 time: 0.170332 Oct 28 09:47:39.411524 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 206/396] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:39 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1506 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:39.566060 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ff271b74-b876-4005-9fbf-3512d8d40569 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:39.574894 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ff271b74-b876-4005-9fbf-3512d8d40569 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.238926 Oct 28 09:47:39.591316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 191/397] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:39 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:40.266280 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-df03c14c-a293-412e-ac5c-d2b5c3847ec6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:40.266927 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-df03c14c-a293-412e-ac5c-d2b5c3847ec6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:40.267287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-00a71433-87d1-4034-a747-786affb39211 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:40.267958 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-00a71433-87d1-4034-a747-786affb39211 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:40.273949 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-df03c14c-a293-412e-ac5c-d2b5c3847ec6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:40.274458 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-df03c14c-a293-412e-ac5c-d2b5c3847ec6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:40.274669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00a71433-87d1-4034-a747-786affb39211 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:40.274897 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-df03c14c-a293-412e-ac5c-d2b5c3847ec6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:40.275145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00a71433-87d1-4034-a747-786affb39211 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:40.275571 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00a71433-87d1-4034-a747-786affb39211 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:40.275666 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-df03c14c-a293-412e-ac5c-d2b5c3847ec6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:40.276044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-df03c14c-a293-412e-ac5c-d2b5c3847ec6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:40.276239 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00a71433-87d1-4034-a747-786affb39211 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:40.276465 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-df03c14c-a293-412e-ac5c-d2b5c3847ec6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:40.276777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00a71433-87d1-4034-a747-786affb39211 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:40.277089 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00a71433-87d1-4034-a747-786affb39211 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:40.467790 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-df03c14c-a293-412e-ac5c-d2b5c3847ec6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:40.475496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-df03c14c-a293-412e-ac5c-d2b5c3847ec6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1371 microversion: 2.1 time: 0.213057 Oct 28 09:47:40.476252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 207/398] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:40 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1371 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:40.480285 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-00a71433-87d1-4034-a747-786affb39211 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:40.488575 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-00a71433-87d1-4034-a747-786affb39211 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:40.490125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-00a71433-87d1-4034-a747-786affb39211 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:40.491391 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-00a71433-87d1-4034-a747-786affb39211 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.228089 Oct 28 09:47:40.491944 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 192/399] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:40 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:40.600776 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8150a119-4095-4a47-96c5-5c6a294a4003 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:40.601457 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8150a119-4095-4a47-96c5-5c6a294a4003 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:40.608886 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8150a119-4095-4a47-96c5-5c6a294a4003 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:40.609387 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8150a119-4095-4a47-96c5-5c6a294a4003 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:40.609821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8150a119-4095-4a47-96c5-5c6a294a4003 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:40.610574 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8150a119-4095-4a47-96c5-5c6a294a4003 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:40.611065 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8150a119-4095-4a47-96c5-5c6a294a4003 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:40.611489 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8150a119-4095-4a47-96c5-5c6a294a4003 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:40.630262 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:40.747420 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-8150a119-4095-4a47-96c5-5c6a294a4003 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:40.756914 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8150a119-4095-4a47-96c5-5c6a294a4003 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.160203 Oct 28 09:47:40.757482 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 208/400] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:40 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:41.497959 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-652cc9ae-dad9-4e9c-a103-949c55e0350c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:41.498705 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-652cc9ae-dad9-4e9c-a103-949c55e0350c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:41.500650 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:41.501344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=10 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:41.508574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-652cc9ae-dad9-4e9c-a103-949c55e0350c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:41.508796 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-652cc9ae-dad9-4e9c-a103-949c55e0350c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:41.509214 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-652cc9ae-dad9-4e9c-a103-949c55e0350c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:41.510119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-652cc9ae-dad9-4e9c-a103-949c55e0350c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:41.510682 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-652cc9ae-dad9-4e9c-a103-949c55e0350c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:41.511100 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-652cc9ae-dad9-4e9c-a103-949c55e0350c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:41.517263 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-29a6c644-9696-43d4-b0e8-a7fb2929e6c3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:41.517866 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-29a6c644-9696-43d4-b0e8-a7fb2929e6c3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:41.525387 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-29a6c644-9696-43d4-b0e8-a7fb2929e6c3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:41.525870 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-29a6c644-9696-43d4-b0e8-a7fb2929e6c3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:41.526319 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-29a6c644-9696-43d4-b0e8-a7fb2929e6c3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:41.527040 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-29a6c644-9696-43d4-b0e8-a7fb2929e6c3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:41.527513 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-29a6c644-9696-43d4-b0e8-a7fb2929e6c3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:41.527929 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-29a6c644-9696-43d4-b0e8-a7fb2929e6c3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:41.720487 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-652cc9ae-dad9-4e9c-a103-949c55e0350c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:41.725813 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-29a6c644-9696-43d4-b0e8-a7fb2929e6c3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:41.729464 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-652cc9ae-dad9-4e9c-a103-949c55e0350c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1371 microversion: 2.1 time: 0.235348 Oct 28 09:47:41.730463 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 193/401] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:41 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1371 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:41.745053 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-29a6c644-9696-43d4-b0e8-a7fb2929e6c3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:41.746524 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-29a6c644-9696-43d4-b0e8-a7fb2929e6c3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:41.747782 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-29a6c644-9696-43d4-b0e8-a7fb2929e6c3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1744 microversion: 2.20 time: 0.234525 Oct 28 09:47:41.748623 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 209/402] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:41 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1744 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:47:41.780739 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e5eb7db0-aa0b-413d-aa17-defa76a799bd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:41.781447 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e5eb7db0-aa0b-413d-aa17-defa76a799bd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:41.787392 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e5eb7db0-aa0b-413d-aa17-defa76a799bd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:41.787969 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e5eb7db0-aa0b-413d-aa17-defa76a799bd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:41.788596 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e5eb7db0-aa0b-413d-aa17-defa76a799bd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:41.789288 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e5eb7db0-aa0b-413d-aa17-defa76a799bd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:41.789735 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e5eb7db0-aa0b-413d-aa17-defa76a799bd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:41.790177 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e5eb7db0-aa0b-413d-aa17-defa76a799bd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:41.923905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e5eb7db0-aa0b-413d-aa17-defa76a799bd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:41.932849 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e5eb7db0-aa0b-413d-aa17-defa76a799bd tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.155988 Oct 28 09:47:41.933417 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 194/403] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:41 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:42.587059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-0f241104-b3f6-4b8d-b91c-40c29b0ffb98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Action: 'create', calling method: .version_select at 0x7f271a455ca0>, body: {"keypair": {"name": "tempest-keypair-624967812", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEmjjRlfp9HPo6j13kwwSvEm7N/QHjewgaqOt+pK553ApAwChzvJFtjpQnr4d+CKqJjB/ispuTBkY4xcAFv/InTpiv261REUXIfYXcxiE75ixPODm10q565COqeRlwu0MA=="}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:42.606381 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-0f241104-b3f6-4b8d-b91c-40c29b0ffb98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for project 4f351d016ace4e369be852bb7aa916fe. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:47:42.610199 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-0f241104-b3f6-4b8d-b91c-40c29b0ffb98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for user e65d298cca0f478eba502edf5bd13954 and project 4f351d016ace4e369be852bb7aa916fe. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:47:42.633368 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-0f241104-b3f6-4b8d-b91c-40c29b0ffb98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.050127 Oct 28 09:47:42.633962 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 210/404] 10.222.0.25 () {60 vars in 1159 bytes} [Fri Oct 28 09:47:42 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:42.756213 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1fa735a7-fe8f-4257-8e1a-10b8623bf5e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:42.756827 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1fa735a7-fe8f-4257-8e1a-10b8623bf5e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:42.764112 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1fa735a7-fe8f-4257-8e1a-10b8623bf5e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:42.764592 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1fa735a7-fe8f-4257-8e1a-10b8623bf5e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:42.765231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1fa735a7-fe8f-4257-8e1a-10b8623bf5e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:42.765924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1fa735a7-fe8f-4257-8e1a-10b8623bf5e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:42.766368 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1fa735a7-fe8f-4257-8e1a-10b8623bf5e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:42.766804 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1fa735a7-fe8f-4257-8e1a-10b8623bf5e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:42.776457 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b7599fe1-59dd-4af6-ba5f-9998141f4880 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:42.777734 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b7599fe1-59dd-4af6-ba5f-9998141f4880 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:42.784014 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:42.787493 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b7599fe1-59dd-4af6-ba5f-9998141f4880 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:42.788730 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b7599fe1-59dd-4af6-ba5f-9998141f4880 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:42.789378 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b7599fe1-59dd-4af6-ba5f-9998141f4880 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:42.790060 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b7599fe1-59dd-4af6-ba5f-9998141f4880 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:42.790555 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b7599fe1-59dd-4af6-ba5f-9998141f4880 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:42.791161 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b7599fe1-59dd-4af6-ba5f-9998141f4880 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:43.055341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1fa735a7-fe8f-4257-8e1a-10b8623bf5e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:43.071197 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b7599fe1-59dd-4af6-ba5f-9998141f4880 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:43.079589 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1fa735a7-fe8f-4257-8e1a-10b8623bf5e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1371 microversion: 2.1 time: 0.324356 Oct 28 09:47:43.103110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 195/405] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:42 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1371 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:43.108263 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b7599fe1-59dd-4af6-ba5f-9998141f4880 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:43.112473 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b7599fe1-59dd-4af6-ba5f-9998141f4880 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:43.114073 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b7599fe1-59dd-4af6-ba5f-9998141f4880 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1744 microversion: 2.20 time: 0.342109 Oct 28 09:47:43.114691 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 211/406] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:42 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1744 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:43.115134 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5c2809c6-37d2-415d-bbe1-bedf9dbea181 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:43.116023 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5c2809c6-37d2-415d-bbe1-bedf9dbea181 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:43.123364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5c2809c6-37d2-415d-bbe1-bedf9dbea181 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:43.124128 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5c2809c6-37d2-415d-bbe1-bedf9dbea181 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:43.125123 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5c2809c6-37d2-415d-bbe1-bedf9dbea181 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:43.126056 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5c2809c6-37d2-415d-bbe1-bedf9dbea181 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:43.126887 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5c2809c6-37d2-415d-bbe1-bedf9dbea181 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:43.128023 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5c2809c6-37d2-415d-bbe1-bedf9dbea181 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:43.356072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5c2809c6-37d2-415d-bbe1-bedf9dbea181 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:43.372021 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5c2809c6-37d2-415d-bbe1-bedf9dbea181 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.266654 Oct 28 09:47:43.373345 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 196/407] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:43 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:44.134338 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-15011863-dd12-49f5-bc86-587853cd0f5d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:44.136347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-15011863-dd12-49f5-bc86-587853cd0f5d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:44.141421 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8fa10cc3-9bd5-4c50-9005-c84d236cab5d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:44.142139 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8fa10cc3-9bd5-4c50-9005-c84d236cab5d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:44.146619 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-15011863-dd12-49f5-bc86-587853cd0f5d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:44.147615 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-15011863-dd12-49f5-bc86-587853cd0f5d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:44.148362 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-15011863-dd12-49f5-bc86-587853cd0f5d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:44.149235 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-15011863-dd12-49f5-bc86-587853cd0f5d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:44.149858 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-15011863-dd12-49f5-bc86-587853cd0f5d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:44.150620 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-15011863-dd12-49f5-bc86-587853cd0f5d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:44.159405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fa10cc3-9bd5-4c50-9005-c84d236cab5d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:44.163897 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fa10cc3-9bd5-4c50-9005-c84d236cab5d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:44.164483 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fa10cc3-9bd5-4c50-9005-c84d236cab5d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:44.165224 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fa10cc3-9bd5-4c50-9005-c84d236cab5d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:44.165983 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fa10cc3-9bd5-4c50-9005-c84d236cab5d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:44.168486 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8fa10cc3-9bd5-4c50-9005-c84d236cab5d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:44.483844 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8fa10cc3-9bd5-4c50-9005-c84d236cab5d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:44.488194 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-15011863-dd12-49f5-bc86-587853cd0f5d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:44.494062 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8fa10cc3-9bd5-4c50-9005-c84d236cab5d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:44.498959 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8fa10cc3-9bd5-4c50-9005-c84d236cab5d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:44.500842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8fa10cc3-9bd5-4c50-9005-c84d236cab5d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1744 microversion: 2.20 time: 0.363450 Oct 28 09:47:44.501406 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 197/408] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:44 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1744 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:47:44.519802 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1539792570", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8cae7e07-cd54-4f98-96b4-a27c37dcb83a"}], "security_groups": [{"name": "tempest-securitygroup--1602960298"}], "key_name": "tempest-keypair-624967812", "user_data": "}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:44.536424 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-15011863-dd12-49f5-bc86-587853cd0f5d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1371 microversion: 2.1 time: 0.406479 Oct 28 09:47:44.538061 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 212/409] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:44 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1371 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:44.554848 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3f043db7-4bad-419d-991f-39775cb56674 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:44.555825 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-3f043db7-4bad-419d-991f-39775cb56674 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:44.562485 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3f043db7-4bad-419d-991f-39775cb56674 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:44.563555 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3f043db7-4bad-419d-991f-39775cb56674 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:44.564156 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3f043db7-4bad-419d-991f-39775cb56674 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:44.565208 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3f043db7-4bad-419d-991f-39775cb56674 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:44.565772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3f043db7-4bad-419d-991f-39775cb56674 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:44.567062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3f043db7-4bad-419d-991f-39775cb56674 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:44.770817 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.network.neutron [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] validate_networks() for [('8cae7e07-cd54-4f98-96b4-a27c37dcb83a', None, None, None, None, None)] {{(pid=127030) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:47:44.824082 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3f043db7-4bad-419d-991f-39775cb56674 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:44.835475 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-3f043db7-4bad-419d-991f-39775cb56674 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.295993 Oct 28 09:47:44.836065 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 213/410] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:44 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:45.442969 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Flavor limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:47:45.443554 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Image limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:47:45.444413 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Flavor pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:47:45.445269 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Image pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:47:45.445904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:47:45.457476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:45.458052 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:45.458988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:45.460372 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:45.461277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:45.462248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:45.484549 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for project 4f351d016ace4e369be852bb7aa916fe. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:47:45.491758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for user e65d298cca0f478eba502edf5bd13954 and project 4f351d016ace4e369be852bb7aa916fe. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:47:45.510853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Going to run 1 instances... {{(pid=127030) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:47:45.519504 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127030) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:47:45.535959 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:45.544359 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6a6beb7e-7d42-4da9-8a7d-593011054150 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:45.545139 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6a6beb7e-7d42-4da9-8a7d-593011054150 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:45.635727 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c8e84c83-a5b5-427d-8edd-023b30cb4f36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.130505 Oct 28 09:47:45.637152 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 198/411] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:47:44 2022] POST /compute/v2.1/servers => generated 402 bytes in 1133 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 09:47:45.665365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6023b9c0-aafe-4608-a722-540dbe616e76 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:45.666755 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6a6beb7e-7d42-4da9-8a7d-593011054150 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:45.668697 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6023b9c0-aafe-4608-a722-540dbe616e76 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:45.668807 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6a6beb7e-7d42-4da9-8a7d-593011054150 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:45.670061 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6a6beb7e-7d42-4da9-8a7d-593011054150 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:45.674214 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6a6beb7e-7d42-4da9-8a7d-593011054150 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:45.675885 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6a6beb7e-7d42-4da9-8a7d-593011054150 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:45.676399 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6a6beb7e-7d42-4da9-8a7d-593011054150 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:45.677387 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6023b9c0-aafe-4608-a722-540dbe616e76 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:45.677884 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6023b9c0-aafe-4608-a722-540dbe616e76 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:45.678328 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6023b9c0-aafe-4608-a722-540dbe616e76 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:45.679116 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6023b9c0-aafe-4608-a722-540dbe616e76 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:45.679572 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6023b9c0-aafe-4608-a722-540dbe616e76 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:45.679999 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6023b9c0-aafe-4608-a722-540dbe616e76 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:45.998141 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6023b9c0-aafe-4608-a722-540dbe616e76 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:46.006263 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6023b9c0-aafe-4608-a722-540dbe616e76 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1371 microversion: 2.1 time: 0.367456 Oct 28 09:47:46.010679 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 199/412] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:45 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1371 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:46.018042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5d131891-3ceb-4158-a3f8-6569d97fd19a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:46.021299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5d131891-3ceb-4158-a3f8-6569d97fd19a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:46.070796 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6a6beb7e-7d42-4da9-8a7d-593011054150 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:46.083963 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6a6beb7e-7d42-4da9-8a7d-593011054150 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:46.093780 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6a6beb7e-7d42-4da9-8a7d-593011054150 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:46.096287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6a6beb7e-7d42-4da9-8a7d-593011054150 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1739 microversion: 2.20 time: 0.558414 Oct 28 09:47:46.097093 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 214/413] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:45 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1739 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:46.107106 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-230540df-caf6-4763-acd3-cd149a617fce tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:46.108137 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-230540df-caf6-4763-acd3-cd149a617fce tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:46.125639 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-230540df-caf6-4763-acd3-cd149a617fce tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:46.126574 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-230540df-caf6-4763-acd3-cd149a617fce tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:46.127251 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-230540df-caf6-4763-acd3-cd149a617fce tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:46.128268 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-230540df-caf6-4763-acd3-cd149a617fce tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:46.128989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-230540df-caf6-4763-acd3-cd149a617fce tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:46.129634 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-230540df-caf6-4763-acd3-cd149a617fce tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:46.163541 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5d131891-3ceb-4158-a3f8-6569d97fd19a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:46.172270 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5d131891-3ceb-4158-a3f8-6569d97fd19a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1241 microversion: 2.1 time: 0.160259 Oct 28 09:47:46.173682 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 200/414] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:46 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1241 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:46.395211 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-230540df-caf6-4763-acd3-cd149a617fce tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:46.408468 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-230540df-caf6-4763-acd3-cd149a617fce tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.310071 Oct 28 09:47:46.409051 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 215/415] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:46 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:46.515506 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-edb56ee3-d4f2-4ba8-a343-02d0eba29226 req-1334eae4-22b8-43e0-883e-cafdaec8378c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c716d5fd-f257-4f6a-8f52-18ad59e5c31e", "name": "network-vif-plugged", "status": "completed", "tag": "c2310b44-82bb-408f-85b5-8a5556b92e25"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:46.540193 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-edb56ee3-d4f2-4ba8-a343-02d0eba29226 req-1334eae4-22b8-43e0-883e-cafdaec8378c service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:46.540964 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-edb56ee3-d4f2-4ba8-a343-02d0eba29226 req-1334eae4-22b8-43e0-883e-cafdaec8378c service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:46.541544 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-edb56ee3-d4f2-4ba8-a343-02d0eba29226 req-1334eae4-22b8-43e0-883e-cafdaec8378c service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:46.580560 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-edb56ee3-d4f2-4ba8-a343-02d0eba29226 req-1334eae4-22b8-43e0-883e-cafdaec8378c service nova] Creating event network-vif-plugged:c2310b44-82bb-408f-85b5-8a5556b92e25 for instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e on hv-cinder-zed-3 Oct 28 09:47:46.600329 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-edb56ee3-d4f2-4ba8-a343-02d0eba29226 req-1334eae4-22b8-43e0-883e-cafdaec8378c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.087279 Oct 28 09:47:46.602235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 201/416] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:47:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:47.035555 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-da2586d5-324b-4e86-ae99-5d8e3c462c2e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:47.036780 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-da2586d5-324b-4e86-ae99-5d8e3c462c2e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:47.043661 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-da2586d5-324b-4e86-ae99-5d8e3c462c2e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:47.044580 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-da2586d5-324b-4e86-ae99-5d8e3c462c2e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:47.045522 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-da2586d5-324b-4e86-ae99-5d8e3c462c2e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:47.046603 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-da2586d5-324b-4e86-ae99-5d8e3c462c2e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:47.047285 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-da2586d5-324b-4e86-ae99-5d8e3c462c2e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:47.047920 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-da2586d5-324b-4e86-ae99-5d8e3c462c2e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:47.136291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3dcf7e1c-6eb7-4c13-80f4-1e4fad063573 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:47.137186 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3dcf7e1c-6eb7-4c13-80f4-1e4fad063573 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:47.149418 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3dcf7e1c-6eb7-4c13-80f4-1e4fad063573 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:47.149902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3dcf7e1c-6eb7-4c13-80f4-1e4fad063573 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:47.150337 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3dcf7e1c-6eb7-4c13-80f4-1e4fad063573 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:47.152015 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3dcf7e1c-6eb7-4c13-80f4-1e4fad063573 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:47.152472 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3dcf7e1c-6eb7-4c13-80f4-1e4fad063573 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:47.154686 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3dcf7e1c-6eb7-4c13-80f4-1e4fad063573 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:47.190715 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:47.296414 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-da2586d5-324b-4e86-ae99-5d8e3c462c2e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:47.306093 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-da2586d5-324b-4e86-ae99-5d8e3c462c2e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1371 microversion: 2.1 time: 0.275278 Oct 28 09:47:47.311751 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 216/417] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:47 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1371 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:47.322243 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5521df56-b016-4a3e-8c2e-117996fa4464 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:47.323118 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5521df56-b016-4a3e-8c2e-117996fa4464 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:47.336210 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5521df56-b016-4a3e-8c2e-117996fa4464 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:47.337509 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5521df56-b016-4a3e-8c2e-117996fa4464 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:47.338184 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5521df56-b016-4a3e-8c2e-117996fa4464 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:47.339247 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5521df56-b016-4a3e-8c2e-117996fa4464 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:47.340054 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5521df56-b016-4a3e-8c2e-117996fa4464 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:47.340683 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5521df56-b016-4a3e-8c2e-117996fa4464 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:47.440237 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3dcf7e1c-6eb7-4c13-80f4-1e4fad063573 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:47.456863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3dcf7e1c-6eb7-4c13-80f4-1e4fad063573 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:47.457281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5521df56-b016-4a3e-8c2e-117996fa4464 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:47.466091 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5521df56-b016-4a3e-8c2e-117996fa4464 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1293 microversion: 2.1 time: 0.152796 Oct 28 09:47:47.467181 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 217/418] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:47 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1293 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:47.471685 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2ab74dfa-1cce-4a8e-bc3c-d2c82ddac125 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:47.472324 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2ab74dfa-1cce-4a8e-bc3c-d2c82ddac125 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:47.475884 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3dcf7e1c-6eb7-4c13-80f4-1e4fad063573 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:47.477286 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3dcf7e1c-6eb7-4c13-80f4-1e4fad063573 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1739 microversion: 2.20 time: 0.357152 Oct 28 09:47:47.477875 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 202/419] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:47 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1739 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:47:47.478359 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2ab74dfa-1cce-4a8e-bc3c-d2c82ddac125 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:47.478883 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2ab74dfa-1cce-4a8e-bc3c-d2c82ddac125 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:47.479320 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2ab74dfa-1cce-4a8e-bc3c-d2c82ddac125 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:47.480017 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2ab74dfa-1cce-4a8e-bc3c-d2c82ddac125 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:47.480466 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2ab74dfa-1cce-4a8e-bc3c-d2c82ddac125 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:47.480888 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2ab74dfa-1cce-4a8e-bc3c-d2c82ddac125 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:47.674318 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2ab74dfa-1cce-4a8e-bc3c-d2c82ddac125 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:47.684560 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2ab74dfa-1cce-4a8e-bc3c-d2c82ddac125 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.216327 Oct 28 09:47:47.685481 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 218/420] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:47 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:48.341119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-133fea0a-30f1-476c-aff7-6f97ccc1bd1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:48.343146 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-133fea0a-30f1-476c-aff7-6f97ccc1bd1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:48.352682 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-133fea0a-30f1-476c-aff7-6f97ccc1bd1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:48.353519 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-133fea0a-30f1-476c-aff7-6f97ccc1bd1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:48.354163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-133fea0a-30f1-476c-aff7-6f97ccc1bd1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:48.355123 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-133fea0a-30f1-476c-aff7-6f97ccc1bd1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:48.355783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-133fea0a-30f1-476c-aff7-6f97ccc1bd1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:48.356396 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-133fea0a-30f1-476c-aff7-6f97ccc1bd1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:48.497226 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8867dd03-22e4-4e9c-bcf4-f57dda2a5f12 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:48.499429 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8867dd03-22e4-4e9c-bcf4-f57dda2a5f12 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:48.508198 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8867dd03-22e4-4e9c-bcf4-f57dda2a5f12 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:48.509690 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8867dd03-22e4-4e9c-bcf4-f57dda2a5f12 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:48.510353 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8867dd03-22e4-4e9c-bcf4-f57dda2a5f12 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:48.511314 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8867dd03-22e4-4e9c-bcf4-f57dda2a5f12 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:48.511916 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8867dd03-22e4-4e9c-bcf4-f57dda2a5f12 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:48.513236 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8867dd03-22e4-4e9c-bcf4-f57dda2a5f12 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:48.619273 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-8867dd03-22e4-4e9c-bcf4-f57dda2a5f12 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:48.629087 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8867dd03-22e4-4e9c-bcf4-f57dda2a5f12 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1299 microversion: 2.1 time: 0.136944 Oct 28 09:47:48.642852 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 219/421] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:48 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1299 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:48.656793 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-133fea0a-30f1-476c-aff7-6f97ccc1bd1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:48.659372 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-eb2be416-c819-458e-8ec4-9597f083f4d4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:48.660988 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-eb2be416-c819-458e-8ec4-9597f083f4d4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:48.672688 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-133fea0a-30f1-476c-aff7-6f97ccc1bd1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1371 microversion: 2.1 time: 0.339706 Oct 28 09:47:48.673827 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eb2be416-c819-458e-8ec4-9597f083f4d4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:48.675064 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eb2be416-c819-458e-8ec4-9597f083f4d4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:48.677139 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eb2be416-c819-458e-8ec4-9597f083f4d4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:48.679732 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eb2be416-c819-458e-8ec4-9597f083f4d4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:48.681013 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eb2be416-c819-458e-8ec4-9597f083f4d4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:48.681671 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eb2be416-c819-458e-8ec4-9597f083f4d4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:48.685434 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 203/422] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:48 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1371 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:48.699526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-a82c1c44-1df5-4974-8507-585434d9af8a req-b2586951-9f55-459e-aa64-ebd3f6c83579 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4891b63b-eb08-448d-a640-f9579c8140ff", "name": "network-vif-plugged", "status": "completed", "tag": "6e78f30d-6812-43c6-910d-70f563ee8ad9"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:48.719117 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-a82c1c44-1df5-4974-8507-585434d9af8a req-b2586951-9f55-459e-aa64-ebd3f6c83579 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:48.720260 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-a82c1c44-1df5-4974-8507-585434d9af8a req-b2586951-9f55-459e-aa64-ebd3f6c83579 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:48.721207 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-a82c1c44-1df5-4974-8507-585434d9af8a req-b2586951-9f55-459e-aa64-ebd3f6c83579 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:48.738325 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-a82c1c44-1df5-4974-8507-585434d9af8a req-b2586951-9f55-459e-aa64-ebd3f6c83579 service nova] Creating event network-vif-plugged:6e78f30d-6812-43c6-910d-70f563ee8ad9 for instance 4891b63b-eb08-448d-a640-f9579c8140ff on hv-cinder-zed-3 Oct 28 09:47:48.749789 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-a82c1c44-1df5-4974-8507-585434d9af8a req-b2586951-9f55-459e-aa64-ebd3f6c83579 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061564 Oct 28 09:47:48.752502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 204/423] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:47:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:48.764942 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3a90d753-5a14-46bd-aa18-2d7d249ec686 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:48.767643 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3a90d753-5a14-46bd-aa18-2d7d249ec686 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:48.777236 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3a90d753-5a14-46bd-aa18-2d7d249ec686 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:48.778071 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3a90d753-5a14-46bd-aa18-2d7d249ec686 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:48.779119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3a90d753-5a14-46bd-aa18-2d7d249ec686 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:48.782741 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3a90d753-5a14-46bd-aa18-2d7d249ec686 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:48.783494 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3a90d753-5a14-46bd-aa18-2d7d249ec686 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:48.784075 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3a90d753-5a14-46bd-aa18-2d7d249ec686 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:49.010864 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-eb2be416-c819-458e-8ec4-9597f083f4d4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:49.023318 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-eb2be416-c819-458e-8ec4-9597f083f4d4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:49.025882 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-eb2be416-c819-458e-8ec4-9597f083f4d4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:49.027264 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-eb2be416-c819-458e-8ec4-9597f083f4d4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1739 microversion: 2.20 time: 0.382784 Oct 28 09:47:49.028316 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 220/424] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:48 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1739 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:49.258183 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3a90d753-5a14-46bd-aa18-2d7d249ec686 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:49.272427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3a90d753-5a14-46bd-aa18-2d7d249ec686 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.514373 Oct 28 09:47:49.274623 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 205/425] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:48 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:49.664232 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c0c041c1-314b-4f46-954e-c4eab8b6ab9c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:49.664910 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c0c041c1-314b-4f46-954e-c4eab8b6ab9c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:49.702254 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a61201df-5801-45e9-a984-35b8f42a62df tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:49.704376 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a61201df-5801-45e9-a984-35b8f42a62df tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:49.709303 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0c041c1-314b-4f46-954e-c4eab8b6ab9c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:49.710996 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0c041c1-314b-4f46-954e-c4eab8b6ab9c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:49.712476 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0c041c1-314b-4f46-954e-c4eab8b6ab9c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:49.715821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0c041c1-314b-4f46-954e-c4eab8b6ab9c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:49.716448 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0c041c1-314b-4f46-954e-c4eab8b6ab9c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:49.717473 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0c041c1-314b-4f46-954e-c4eab8b6ab9c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:49.722846 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a61201df-5801-45e9-a984-35b8f42a62df tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:49.729965 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a61201df-5801-45e9-a984-35b8f42a62df tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:49.731838 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a61201df-5801-45e9-a984-35b8f42a62df tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:49.736339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a61201df-5801-45e9-a984-35b8f42a62df tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:49.737183 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a61201df-5801-45e9-a984-35b8f42a62df tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:49.740026 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a61201df-5801-45e9-a984-35b8f42a62df tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:50.124281 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a61201df-5801-45e9-a984-35b8f42a62df tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:50.126895 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c0c041c1-314b-4f46-954e-c4eab8b6ab9c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:50.133947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a61201df-5801-45e9-a984-35b8f42a62df tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1371 microversion: 2.1 time: 0.437131 Oct 28 09:47:50.134996 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c0c041c1-314b-4f46-954e-c4eab8b6ab9c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1367 microversion: 2.1 time: 0.475484 Oct 28 09:47:50.135559 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 221/426] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:49 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1367 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:50.139984 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-93e82aae-8e39-4a45-b392-07fd8cbff93d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:50.140568 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-93e82aae-8e39-4a45-b392-07fd8cbff93d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:50.145152 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 206/427] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:49 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1371 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:50.163809 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-93e82aae-8e39-4a45-b392-07fd8cbff93d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:50.164692 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-93e82aae-8e39-4a45-b392-07fd8cbff93d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:50.165584 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-93e82aae-8e39-4a45-b392-07fd8cbff93d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:50.166754 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-93e82aae-8e39-4a45-b392-07fd8cbff93d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:50.167283 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-93e82aae-8e39-4a45-b392-07fd8cbff93d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:50.168197 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-93e82aae-8e39-4a45-b392-07fd8cbff93d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:50.298737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-686f9975-243a-42b5-aaa2-088335d68bf7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:50.299475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-686f9975-243a-42b5-aaa2-088335d68bf7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:50.306929 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-686f9975-243a-42b5-aaa2-088335d68bf7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:50.307810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-686f9975-243a-42b5-aaa2-088335d68bf7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:50.308606 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-686f9975-243a-42b5-aaa2-088335d68bf7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:50.309535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-686f9975-243a-42b5-aaa2-088335d68bf7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:50.310199 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-686f9975-243a-42b5-aaa2-088335d68bf7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:50.311002 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-686f9975-243a-42b5-aaa2-088335d68bf7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:50.782781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-686f9975-243a-42b5-aaa2-088335d68bf7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:50.788505 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-93e82aae-8e39-4a45-b392-07fd8cbff93d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:50.797288 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-686f9975-243a-42b5-aaa2-088335d68bf7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.502872 Oct 28 09:47:50.798460 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 207/428] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:50 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:50.803238 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-93e82aae-8e39-4a45-b392-07fd8cbff93d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:50.805019 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-93e82aae-8e39-4a45-b392-07fd8cbff93d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:50.812155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-93e82aae-8e39-4a45-b392-07fd8cbff93d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1739 microversion: 2.20 time: 0.675442 Oct 28 09:47:50.813196 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 222/429] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:50 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1739 bytes in 677 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:51.171888 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-06190d81-b351-44e2-8ff1-819bd6f36df7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:51.174098 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-74f7193d-23b9-46ed-9926-2138709dc967 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:51.176369 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-74f7193d-23b9-46ed-9926-2138709dc967 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:51.182286 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-06190d81-b351-44e2-8ff1-819bd6f36df7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:51.199971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-74f7193d-23b9-46ed-9926-2138709dc967 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:51.201560 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-74f7193d-23b9-46ed-9926-2138709dc967 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:51.202817 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-06190d81-b351-44e2-8ff1-819bd6f36df7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:51.203566 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-74f7193d-23b9-46ed-9926-2138709dc967 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:51.204543 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-74f7193d-23b9-46ed-9926-2138709dc967 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:51.205505 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-06190d81-b351-44e2-8ff1-819bd6f36df7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:51.206178 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-06190d81-b351-44e2-8ff1-819bd6f36df7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:51.207091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-74f7193d-23b9-46ed-9926-2138709dc967 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:51.208348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-74f7193d-23b9-46ed-9926-2138709dc967 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:51.213087 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-06190d81-b351-44e2-8ff1-819bd6f36df7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:51.233288 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-06190d81-b351-44e2-8ff1-819bd6f36df7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:51.235054 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-06190d81-b351-44e2-8ff1-819bd6f36df7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:51.512990 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-74f7193d-23b9-46ed-9926-2138709dc967 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:51.532954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-74f7193d-23b9-46ed-9926-2138709dc967 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1565 microversion: 2.1 time: 0.377737 Oct 28 09:47:51.564938 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 208/430] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:51 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1565 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:51.584064 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-d7436b20-eca2-4f4f-a859-6576d2e4d78d req-e165a712-9906-415a-a03a-dde985e111f5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f8edd976-e011-4194-ad94-1160cc01c63b", "tag": "450bfe2d-019d-4361-97e8-cd9a4452887d"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:51.603580 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-d7436b20-eca2-4f4f-a859-6576d2e4d78d req-e165a712-9906-415a-a03a-dde985e111f5 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:51.604369 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-d7436b20-eca2-4f4f-a859-6576d2e4d78d req-e165a712-9906-415a-a03a-dde985e111f5 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:51.604810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-d7436b20-eca2-4f4f-a859-6576d2e4d78d req-e165a712-9906-415a-a03a-dde985e111f5 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:51.619021 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-d7436b20-eca2-4f4f-a859-6576d2e4d78d req-e165a712-9906-415a-a03a-dde985e111f5 service nova] Creating event network-changed:450bfe2d-019d-4361-97e8-cd9a4452887d for instance f8edd976-e011-4194-ad94-1160cc01c63b on hv-cinder-zed-3 Oct 28 09:47:51.624493 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-d7436b20-eca2-4f4f-a859-6576d2e4d78d req-e165a712-9906-415a-a03a-dde985e111f5 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057663 Oct 28 09:47:51.625383 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 209/431] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:47:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:51.631999 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f7932cc3-d578-48b9-8f66-f22d52025692 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:51.632647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f7932cc3-d578-48b9-8f66-f22d52025692 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:51.639367 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f7932cc3-d578-48b9-8f66-f22d52025692 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:51.639855 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f7932cc3-d578-48b9-8f66-f22d52025692 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:51.640344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f7932cc3-d578-48b9-8f66-f22d52025692 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:51.641061 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f7932cc3-d578-48b9-8f66-f22d52025692 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:51.641507 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f7932cc3-d578-48b9-8f66-f22d52025692 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:51.641976 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f7932cc3-d578-48b9-8f66-f22d52025692 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:51.655239 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-06190d81-b351-44e2-8ff1-819bd6f36df7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:51.667468 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-06190d81-b351-44e2-8ff1-819bd6f36df7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1367 microversion: 2.1 time: 0.504210 Oct 28 09:47:51.678782 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 223/432] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:51 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1367 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:51.756482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f7932cc3-d578-48b9-8f66-f22d52025692 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.130056 Oct 28 09:47:51.757242 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 210/433] 10.222.0.25 () {58 vars in 1275 bytes} [Fri Oct 28 09:47:51 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-interface => generated 284 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:51.829542 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5a3711c8-c4b5-4f9b-bbf5-b3082a8ce0c2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:51.830752 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5a3711c8-c4b5-4f9b-bbf5-b3082a8ce0c2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:51.839269 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6a088a5c-51d1-4c8e-8049-49214e4b71ee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:51.840455 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6a088a5c-51d1-4c8e-8049-49214e4b71ee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:51.860810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5a3711c8-c4b5-4f9b-bbf5-b3082a8ce0c2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:51.863126 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5a3711c8-c4b5-4f9b-bbf5-b3082a8ce0c2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:51.863721 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6a088a5c-51d1-4c8e-8049-49214e4b71ee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:51.867126 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6a088a5c-51d1-4c8e-8049-49214e4b71ee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:51.867585 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6a088a5c-51d1-4c8e-8049-49214e4b71ee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:51.868321 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6a088a5c-51d1-4c8e-8049-49214e4b71ee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:51.868771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6a088a5c-51d1-4c8e-8049-49214e4b71ee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:51.869221 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6a088a5c-51d1-4c8e-8049-49214e4b71ee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:51.873666 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5a3711c8-c4b5-4f9b-bbf5-b3082a8ce0c2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:51.874762 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5a3711c8-c4b5-4f9b-bbf5-b3082a8ce0c2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:51.875402 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5a3711c8-c4b5-4f9b-bbf5-b3082a8ce0c2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:51.875858 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5a3711c8-c4b5-4f9b-bbf5-b3082a8ce0c2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:52.225134 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6a088a5c-51d1-4c8e-8049-49214e4b71ee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:52.234802 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6a088a5c-51d1-4c8e-8049-49214e4b71ee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:52.242644 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6a088a5c-51d1-4c8e-8049-49214e4b71ee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:52.244106 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6a088a5c-51d1-4c8e-8049-49214e4b71ee tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1739 microversion: 2.20 time: 0.408552 Oct 28 09:47:52.249474 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 211/434] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:51 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1739 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:47:52.337110 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5a3711c8-c4b5-4f9b-bbf5-b3082a8ce0c2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:52.348857 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5a3711c8-c4b5-4f9b-bbf5-b3082a8ce0c2 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.523634 Oct 28 09:47:52.366617 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 224/435] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:51 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:52.692462 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5e4e3d38-6043-45e8-8f2a-5ae115162537 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:52.725515 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5e4e3d38-6043-45e8-8f2a-5ae115162537 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:52.737412 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e4e3d38-6043-45e8-8f2a-5ae115162537 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:52.738474 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e4e3d38-6043-45e8-8f2a-5ae115162537 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:52.739296 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e4e3d38-6043-45e8-8f2a-5ae115162537 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:52.740301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e4e3d38-6043-45e8-8f2a-5ae115162537 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:52.740992 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e4e3d38-6043-45e8-8f2a-5ae115162537 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:52.741697 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e4e3d38-6043-45e8-8f2a-5ae115162537 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:53.169829 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-18b53856-920a-4db7-9127-76e09f571a93 req-b933cbf2-2c6b-4de6-af7a-7ed0de2177a6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4891b63b-eb08-448d-a640-f9579c8140ff", "tag": "6e78f30d-6812-43c6-910d-70f563ee8ad9"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:53.184971 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=4 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:53.189144 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-18b53856-920a-4db7-9127-76e09f571a93 req-b933cbf2-2c6b-4de6-af7a-7ed0de2177a6 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:53.192715 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-18b53856-920a-4db7-9127-76e09f571a93 req-b933cbf2-2c6b-4de6-af7a-7ed0de2177a6 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:53.193534 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-18b53856-920a-4db7-9127-76e09f571a93 req-b933cbf2-2c6b-4de6-af7a-7ed0de2177a6 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:53.227184 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5e4e3d38-6043-45e8-8f2a-5ae115162537 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:53.231320 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-18b53856-920a-4db7-9127-76e09f571a93 req-b933cbf2-2c6b-4de6-af7a-7ed0de2177a6 service nova] Creating event network-changed:6e78f30d-6812-43c6-910d-70f563ee8ad9 for instance 4891b63b-eb08-448d-a640-f9579c8140ff on hv-cinder-zed-3 Oct 28 09:47:53.237945 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-18b53856-920a-4db7-9127-76e09f571a93 req-b933cbf2-2c6b-4de6-af7a-7ed0de2177a6 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071964 Oct 28 09:47:53.238988 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 225/436] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:47:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:47:53.240590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5e4e3d38-6043-45e8-8f2a-5ae115162537 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1367 microversion: 2.1 time: 0.552393 Oct 28 09:47:53.242427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 212/437] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:52 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1367 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:53.275642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-dbb28fee-ff15-4e47-a4ba-be857419bf24 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:53.276333 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-dbb28fee-ff15-4e47-a4ba-be857419bf24 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:53.304842 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-dbb28fee-ff15-4e47-a4ba-be857419bf24 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:53.305332 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-dbb28fee-ff15-4e47-a4ba-be857419bf24 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:53.305774 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-dbb28fee-ff15-4e47-a4ba-be857419bf24 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:53.307049 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-dbb28fee-ff15-4e47-a4ba-be857419bf24 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:53.307723 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-dbb28fee-ff15-4e47-a4ba-be857419bf24 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:53.308323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-dbb28fee-ff15-4e47-a4ba-be857419bf24 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:53.377268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-95fd2714-db9e-4473-9dad-2c6ae1d1e2c0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:53.377880 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-95fd2714-db9e-4473-9dad-2c6ae1d1e2c0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:53.387327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95fd2714-db9e-4473-9dad-2c6ae1d1e2c0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:53.387821 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95fd2714-db9e-4473-9dad-2c6ae1d1e2c0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:53.388252 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95fd2714-db9e-4473-9dad-2c6ae1d1e2c0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:53.388962 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95fd2714-db9e-4473-9dad-2c6ae1d1e2c0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:53.389417 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95fd2714-db9e-4473-9dad-2c6ae1d1e2c0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:53.389836 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95fd2714-db9e-4473-9dad-2c6ae1d1e2c0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:53.974008 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-dbb28fee-ff15-4e47-a4ba-be857419bf24 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:53.980545 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:54.000787 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-95fd2714-db9e-4473-9dad-2c6ae1d1e2c0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:54.019602 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-95fd2714-db9e-4473-9dad-2c6ae1d1e2c0 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.646162 Oct 28 09:47:54.029558 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 213/438] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:53 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:54.080225 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-dbb28fee-ff15-4e47-a4ba-be857419bf24 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:54.086863 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-dbb28fee-ff15-4e47-a4ba-be857419bf24 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:54.088477 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-dbb28fee-ff15-4e47-a4ba-be857419bf24 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1739 microversion: 2.20 time: 0.817090 Oct 28 09:47:54.089201 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 226/439] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:53 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1739 bytes in 819 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:47:54.269331 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d7c2668f-086d-4cec-90e7-679fbb77003b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:54.269956 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d7c2668f-086d-4cec-90e7-679fbb77003b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:54.277490 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d7c2668f-086d-4cec-90e7-679fbb77003b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:54.277981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d7c2668f-086d-4cec-90e7-679fbb77003b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:54.278821 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d7c2668f-086d-4cec-90e7-679fbb77003b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:54.279866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d7c2668f-086d-4cec-90e7-679fbb77003b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:54.280475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d7c2668f-086d-4cec-90e7-679fbb77003b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:54.281288 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d7c2668f-086d-4cec-90e7-679fbb77003b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:54.513628 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d7c2668f-086d-4cec-90e7-679fbb77003b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:54.530378 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d7c2668f-086d-4cec-90e7-679fbb77003b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1367 microversion: 2.1 time: 0.265260 Oct 28 09:47:54.531483 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 214/440] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:54 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1367 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:55.066301 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-169c1f61-e3ed-41d6-b560-6ba10ccd6464 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:55.066844 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-169c1f61-e3ed-41d6-b560-6ba10ccd6464 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:55.073548 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-169c1f61-e3ed-41d6-b560-6ba10ccd6464 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:55.073964 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-169c1f61-e3ed-41d6-b560-6ba10ccd6464 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:55.075103 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-169c1f61-e3ed-41d6-b560-6ba10ccd6464 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:55.075896 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-169c1f61-e3ed-41d6-b560-6ba10ccd6464 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:55.076409 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-169c1f61-e3ed-41d6-b560-6ba10ccd6464 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:55.076906 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-169c1f61-e3ed-41d6-b560-6ba10ccd6464 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:55.131244 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a8f3fff9-ef30-4865-bc28-6aa5bcac163c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:55.132653 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a8f3fff9-ef30-4865-bc28-6aa5bcac163c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:55.155582 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a8f3fff9-ef30-4865-bc28-6aa5bcac163c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:55.159411 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a8f3fff9-ef30-4865-bc28-6aa5bcac163c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:55.166536 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a8f3fff9-ef30-4865-bc28-6aa5bcac163c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:55.167608 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a8f3fff9-ef30-4865-bc28-6aa5bcac163c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:55.168461 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a8f3fff9-ef30-4865-bc28-6aa5bcac163c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:55.169051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a8f3fff9-ef30-4865-bc28-6aa5bcac163c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:55.414435 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-169c1f61-e3ed-41d6-b560-6ba10ccd6464 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:55.425358 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-169c1f61-e3ed-41d6-b560-6ba10ccd6464 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1723 microversion: 2.1 time: 0.362780 Oct 28 09:47:55.439912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 227/441] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:55 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1723 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:55.453729 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a8f3fff9-ef30-4865-bc28-6aa5bcac163c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:55.473081 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a8f3fff9-ef30-4865-bc28-6aa5bcac163c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:55.474564 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a8f3fff9-ef30-4865-bc28-6aa5bcac163c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:55.475737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a8f3fff9-ef30-4865-bc28-6aa5bcac163c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1739 microversion: 2.20 time: 0.349486 Oct 28 09:47:55.486430 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 215/442] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:55 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1739 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:47:55.567462 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b78a94f1-6d8f-44c8-af73-86dbcce67b7a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:55.568077 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b78a94f1-6d8f-44c8-af73-86dbcce67b7a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:55.574281 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b78a94f1-6d8f-44c8-af73-86dbcce67b7a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:55.574814 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b78a94f1-6d8f-44c8-af73-86dbcce67b7a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:55.575556 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b78a94f1-6d8f-44c8-af73-86dbcce67b7a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:55.575960 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b78a94f1-6d8f-44c8-af73-86dbcce67b7a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:55.576588 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b78a94f1-6d8f-44c8-af73-86dbcce67b7a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:55.577163 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b78a94f1-6d8f-44c8-af73-86dbcce67b7a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:55.916727 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b78a94f1-6d8f-44c8-af73-86dbcce67b7a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:55.927707 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b78a94f1-6d8f-44c8-af73-86dbcce67b7a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1367 microversion: 2.1 time: 0.364433 Oct 28 09:47:55.929036 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 228/443] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:55 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1367 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:56.452189 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-479c70f3-2be3-43e1-b3c3-9a0d8d1237a4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:56.452780 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-479c70f3-2be3-43e1-b3c3-9a0d8d1237a4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:56.454747 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:56.455872 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=8 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:56.462302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-479c70f3-2be3-43e1-b3c3-9a0d8d1237a4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:56.462840 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-479c70f3-2be3-43e1-b3c3-9a0d8d1237a4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:56.463294 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-479c70f3-2be3-43e1-b3c3-9a0d8d1237a4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:56.467536 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-479c70f3-2be3-43e1-b3c3-9a0d8d1237a4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:56.468324 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-479c70f3-2be3-43e1-b3c3-9a0d8d1237a4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:56.468830 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-479c70f3-2be3-43e1-b3c3-9a0d8d1237a4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:56.508914 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-0dce2696-ab02-4085-8684-6bd5089f887d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:56.510276 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-0dce2696-ab02-4085-8684-6bd5089f887d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:56.517471 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0dce2696-ab02-4085-8684-6bd5089f887d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:56.518325 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0dce2696-ab02-4085-8684-6bd5089f887d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:56.523466 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0dce2696-ab02-4085-8684-6bd5089f887d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:56.524797 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0dce2696-ab02-4085-8684-6bd5089f887d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:56.526248 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0dce2696-ab02-4085-8684-6bd5089f887d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:56.526939 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0dce2696-ab02-4085-8684-6bd5089f887d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:56.688925 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-479c70f3-2be3-43e1-b3c3-9a0d8d1237a4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:56.697642 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-479c70f3-2be3-43e1-b3c3-9a0d8d1237a4 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1700 microversion: 2.1 time: 0.249533 Oct 28 09:47:56.698225 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 216/444] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:56 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1700 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:56.725945 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5f154c40-2341-474a-8252-196e80fd4ad6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:56.733828 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f154c40-2341-474a-8252-196e80fd4ad6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:56.734641 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f154c40-2341-474a-8252-196e80fd4ad6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:56.735076 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f154c40-2341-474a-8252-196e80fd4ad6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:56.762796 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5f154c40-2341-474a-8252-196e80fd4ad6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Going to try to start instance {{(pid=127030) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 28 09:47:56.803999 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0dce2696-ab02-4085-8684-6bd5089f887d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:56.816660 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0dce2696-ab02-4085-8684-6bd5089f887d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:56.818466 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0dce2696-ab02-4085-8684-6bd5089f887d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:56.824373 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-0dce2696-ab02-4085-8684-6bd5089f887d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1739 microversion: 2.20 time: 0.325460 Oct 28 09:47:56.836586 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 229/445] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:56 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1739 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:56.841271 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5f154c40-2341-474a-8252-196e80fd4ad6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "POST /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/action" status: 202 len: 0 microversion: 2.1 time: 0.119549 Oct 28 09:47:56.841882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 217/446] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:47:56 2022] POST /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/action => generated 0 bytes in 121 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 09:47:56.851595 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-519adf1a-5530-4089-957f-77678e003e5f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:56.852459 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-519adf1a-5530-4089-957f-77678e003e5f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:56.861024 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-519adf1a-5530-4089-957f-77678e003e5f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:56.862327 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-519adf1a-5530-4089-957f-77678e003e5f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:56.863318 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-519adf1a-5530-4089-957f-77678e003e5f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:56.864544 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-519adf1a-5530-4089-957f-77678e003e5f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:56.865133 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-519adf1a-5530-4089-957f-77678e003e5f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:56.865567 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-519adf1a-5530-4089-957f-77678e003e5f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:56.952725 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2e824c6e-57ca-41eb-8b7f-4d71ccb737d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:56.953754 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2e824c6e-57ca-41eb-8b7f-4d71ccb737d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:56.963501 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2e824c6e-57ca-41eb-8b7f-4d71ccb737d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:56.964348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2e824c6e-57ca-41eb-8b7f-4d71ccb737d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:56.964974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2e824c6e-57ca-41eb-8b7f-4d71ccb737d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:56.965907 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2e824c6e-57ca-41eb-8b7f-4d71ccb737d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:56.966580 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2e824c6e-57ca-41eb-8b7f-4d71ccb737d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:56.967189 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2e824c6e-57ca-41eb-8b7f-4d71ccb737d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:57.128817 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-519adf1a-5530-4089-957f-77678e003e5f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:57.137524 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-519adf1a-5530-4089-957f-77678e003e5f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1709 microversion: 2.1 time: 0.289977 Oct 28 09:47:57.138300 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 230/447] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:56 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1709 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:57.172391 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2e824c6e-57ca-41eb-8b7f-4d71ccb737d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:57.181517 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2e824c6e-57ca-41eb-8b7f-4d71ccb737d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1367 microversion: 2.1 time: 0.233403 Oct 28 09:47:57.183614 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 218/448] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:56 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1367 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:57.853161 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2e47d147-cbc7-464d-9e75-215c07b7f46f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:57.853862 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2e47d147-cbc7-464d-9e75-215c07b7f46f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:57.871017 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2e47d147-cbc7-464d-9e75-215c07b7f46f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:57.871578 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2e47d147-cbc7-464d-9e75-215c07b7f46f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:57.873254 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2e47d147-cbc7-464d-9e75-215c07b7f46f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:57.874048 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2e47d147-cbc7-464d-9e75-215c07b7f46f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:57.875226 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2e47d147-cbc7-464d-9e75-215c07b7f46f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:57.876634 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2e47d147-cbc7-464d-9e75-215c07b7f46f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:58.166309 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-79c739c7-f0f8-43cf-942e-fa26d6905648 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:58.167114 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-79c739c7-f0f8-43cf-942e-fa26d6905648 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:58.173834 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-79c739c7-f0f8-43cf-942e-fa26d6905648 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:58.174722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-79c739c7-f0f8-43cf-942e-fa26d6905648 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:58.175956 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-79c739c7-f0f8-43cf-942e-fa26d6905648 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:58.176673 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-79c739c7-f0f8-43cf-942e-fa26d6905648 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:58.177835 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-79c739c7-f0f8-43cf-942e-fa26d6905648 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:58.177835 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-79c739c7-f0f8-43cf-942e-fa26d6905648 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:58.384769 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2e47d147-cbc7-464d-9e75-215c07b7f46f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:58.393366 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2e47d147-cbc7-464d-9e75-215c07b7f46f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:58.400592 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2e47d147-cbc7-464d-9e75-215c07b7f46f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:58.401949 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2e47d147-cbc7-464d-9e75-215c07b7f46f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1739 microversion: 2.20 time: 0.553140 Oct 28 09:47:58.405999 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 231/449] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:57 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1739 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:58.411608 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b5d5e5b2-cc51-412c-a5c4-1a9599577fa8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:58.416377 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b5d5e5b2-cc51-412c-a5c4-1a9599577fa8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:58.431976 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b5d5e5b2-cc51-412c-a5c4-1a9599577fa8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:58.435631 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b5d5e5b2-cc51-412c-a5c4-1a9599577fa8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:58.437438 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b5d5e5b2-cc51-412c-a5c4-1a9599577fa8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:58.440033 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b5d5e5b2-cc51-412c-a5c4-1a9599577fa8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:58.444295 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b5d5e5b2-cc51-412c-a5c4-1a9599577fa8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:58.444805 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b5d5e5b2-cc51-412c-a5c4-1a9599577fa8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:58.537711 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-79c739c7-f0f8-43cf-942e-fa26d6905648 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:58.554528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:58.559936 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-79c739c7-f0f8-43cf-942e-fa26d6905648 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1709 microversion: 2.1 time: 0.398315 Oct 28 09:47:58.587981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 219/450] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:58 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1709 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:58.606144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-d526c5c7-711b-4484-9d01-012c1b16708e req-41121f83-71b7-4b1b-8e12-07805972fe03 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f8edd976-e011-4194-ad94-1160cc01c63b", "name": "network-vif-plugged", "status": "completed", "tag": "450bfe2d-019d-4361-97e8-cd9a4452887d"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:58.632616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-d526c5c7-711b-4484-9d01-012c1b16708e req-41121f83-71b7-4b1b-8e12-07805972fe03 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:58.633466 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-d526c5c7-711b-4484-9d01-012c1b16708e req-41121f83-71b7-4b1b-8e12-07805972fe03 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:58.634843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-d526c5c7-711b-4484-9d01-012c1b16708e req-41121f83-71b7-4b1b-8e12-07805972fe03 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:58.654042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-d526c5c7-711b-4484-9d01-012c1b16708e req-41121f83-71b7-4b1b-8e12-07805972fe03 service nova] Creating event network-vif-plugged:450bfe2d-019d-4361-97e8-cd9a4452887d for instance f8edd976-e011-4194-ad94-1160cc01c63b on hv-cinder-zed-3 Oct 28 09:47:58.663176 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-d526c5c7-711b-4484-9d01-012c1b16708e req-41121f83-71b7-4b1b-8e12-07805972fe03 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066837 Oct 28 09:47:58.664809 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 220/451] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:47:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:47:58.704941 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b5d5e5b2-cc51-412c-a5c4-1a9599577fa8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:58.714239 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b5d5e5b2-cc51-412c-a5c4-1a9599577fa8 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1367 microversion: 2.1 time: 0.307089 Oct 28 09:47:58.715555 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 232/452] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:58 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1367 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:47:59.448682 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e773b8a3-2f73-4532-8aba-349b43fe0af8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:59.449283 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e773b8a3-2f73-4532-8aba-349b43fe0af8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:59.455398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e773b8a3-2f73-4532-8aba-349b43fe0af8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:59.455877 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e773b8a3-2f73-4532-8aba-349b43fe0af8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:59.456306 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e773b8a3-2f73-4532-8aba-349b43fe0af8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:59.456992 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e773b8a3-2f73-4532-8aba-349b43fe0af8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:59.457466 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e773b8a3-2f73-4532-8aba-349b43fe0af8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:59.457887 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e773b8a3-2f73-4532-8aba-349b43fe0af8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:59.592337 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-bded8dad-d556-4dfd-ae7f-1add7c2a7509 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:59.593413 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-bded8dad-d556-4dfd-ae7f-1add7c2a7509 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:59.601564 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e773b8a3-2f73-4532-8aba-349b43fe0af8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:59.601808 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bded8dad-d556-4dfd-ae7f-1add7c2a7509 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:59.602363 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bded8dad-d556-4dfd-ae7f-1add7c2a7509 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:59.602957 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bded8dad-d556-4dfd-ae7f-1add7c2a7509 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:59.603762 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bded8dad-d556-4dfd-ae7f-1add7c2a7509 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:59.604302 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bded8dad-d556-4dfd-ae7f-1add7c2a7509 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:59.604813 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bded8dad-d556-4dfd-ae7f-1add7c2a7509 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:59.610724 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e773b8a3-2f73-4532-8aba-349b43fe0af8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:59.612177 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e773b8a3-2f73-4532-8aba-349b43fe0af8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:59.613389 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e773b8a3-2f73-4532-8aba-349b43fe0af8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1739 microversion: 2.20 time: 0.169635 Oct 28 09:47:59.614051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 221/453] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:47:59 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1739 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:47:59.618946 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7aefac91-0d9d-4613-ab43-f51acd4d33a0 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f68b5d8c-4184-4295-98e7-4b8b535e632b", "device": "/dev/sdb"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:47:59.621778 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7aefac91-0d9d-4613-ab43-f51acd4d33a0 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:59.628349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7aefac91-0d9d-4613-ab43-f51acd4d33a0 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:59.629032 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7aefac91-0d9d-4613-ab43-f51acd4d33a0 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:59.629632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7aefac91-0d9d-4613-ab43-f51acd4d33a0 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:59.630520 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7aefac91-0d9d-4613-ab43-f51acd4d33a0 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:59.631127 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7aefac91-0d9d-4613-ab43-f51acd4d33a0 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:59.631731 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7aefac91-0d9d-4613-ab43-f51acd4d33a0 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:59.760268 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-bded8dad-d556-4dfd-ae7f-1add7c2a7509 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:59.769402 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-bded8dad-d556-4dfd-ae7f-1add7c2a7509 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1709 microversion: 2.1 time: 0.181328 Oct 28 09:47:59.769973 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 233/454] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:59 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1709 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:47:59.774833 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c0a83804-aa25-4f9e-b234-340e86d872ed tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:47:59.775440 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c0a83804-aa25-4f9e-b234-340e86d872ed tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:47:59.781092 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0a83804-aa25-4f9e-b234-340e86d872ed tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:59.781579 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0a83804-aa25-4f9e-b234-340e86d872ed tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:59.782015 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0a83804-aa25-4f9e-b234-340e86d872ed tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:59.782739 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0a83804-aa25-4f9e-b234-340e86d872ed tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:47:59.783189 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0a83804-aa25-4f9e-b234-340e86d872ed tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:47:59.783614 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c0a83804-aa25-4f9e-b234-340e86d872ed tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:47:59.916068 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c0a83804-aa25-4f9e-b234-340e86d872ed tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:47:59.924124 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c0a83804-aa25-4f9e-b234-340e86d872ed tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1367 microversion: 2.1 time: 0.152967 Oct 28 09:47:59.924732 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 234/455] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:47:59 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1367 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:00.639376 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-0ad83de5-61b6-4520-8e98-6c8da2881e96 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:00.640044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-0ad83de5-61b6-4520-8e98-6c8da2881e96 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:00.646237 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ad83de5-61b6-4520-8e98-6c8da2881e96 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:00.646758 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ad83de5-61b6-4520-8e98-6c8da2881e96 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:00.647184 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ad83de5-61b6-4520-8e98-6c8da2881e96 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:00.647863 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ad83de5-61b6-4520-8e98-6c8da2881e96 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:00.648307 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ad83de5-61b6-4520-8e98-6c8da2881e96 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:00.648721 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ad83de5-61b6-4520-8e98-6c8da2881e96 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:00.820681 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0ad83de5-61b6-4520-8e98-6c8da2881e96 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:00.829174 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0ad83de5-61b6-4520-8e98-6c8da2881e96 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:00.830577 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0ad83de5-61b6-4520-8e98-6c8da2881e96 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:00.832967 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-0ad83de5-61b6-4520-8e98-6c8da2881e96 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1721 microversion: 2.20 time: 0.196294 Oct 28 09:48:00.833835 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 235/456] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:00 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1721 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:00.838198 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-aabe3266-0b17-41f9-b32a-90ad48834985 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:00.839147 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-aabe3266-0b17-41f9-b32a-90ad48834985 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:00.844818 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aabe3266-0b17-41f9-b32a-90ad48834985 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:00.845277 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aabe3266-0b17-41f9-b32a-90ad48834985 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:00.845704 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aabe3266-0b17-41f9-b32a-90ad48834985 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:00.846377 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aabe3266-0b17-41f9-b32a-90ad48834985 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:00.847383 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aabe3266-0b17-41f9-b32a-90ad48834985 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:00.847799 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aabe3266-0b17-41f9-b32a-90ad48834985 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:00.994642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-aabe3266-0b17-41f9-b32a-90ad48834985 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:01.004181 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-aabe3266-0b17-41f9-b32a-90ad48834985 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1713 microversion: 2.1 time: 0.169271 Oct 28 09:48:01.004767 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 236/457] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:00 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1713 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:01.009592 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-9fec6764-de0d-4eb4-aadb-2a382e5832fa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:01.010216 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-9fec6764-de0d-4eb4-aadb-2a382e5832fa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:01.016396 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9fec6764-de0d-4eb4-aadb-2a382e5832fa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:01.016893 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9fec6764-de0d-4eb4-aadb-2a382e5832fa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:01.017331 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9fec6764-de0d-4eb4-aadb-2a382e5832fa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:01.018042 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9fec6764-de0d-4eb4-aadb-2a382e5832fa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:01.018542 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9fec6764-de0d-4eb4-aadb-2a382e5832fa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:01.019528 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9fec6764-de0d-4eb4-aadb-2a382e5832fa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:01.159772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-9fec6764-de0d-4eb4-aadb-2a382e5832fa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:01.167440 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-9fec6764-de0d-4eb4-aadb-2a382e5832fa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1367 microversion: 2.1 time: 0.161726 Oct 28 09:48:01.168309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 237/458] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:01 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1367 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:01.859410 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-73bde7f9-bf69-4cd5-a535-83ef2d458d67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:01.860823 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-73bde7f9-bf69-4cd5-a535-83ef2d458d67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:01.868680 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73bde7f9-bf69-4cd5-a535-83ef2d458d67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:01.869242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73bde7f9-bf69-4cd5-a535-83ef2d458d67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:01.869766 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73bde7f9-bf69-4cd5-a535-83ef2d458d67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:01.870617 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73bde7f9-bf69-4cd5-a535-83ef2d458d67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:01.871177 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73bde7f9-bf69-4cd5-a535-83ef2d458d67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:01.871598 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-73bde7f9-bf69-4cd5-a535-83ef2d458d67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:02.018355 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-73bde7f9-bf69-4cd5-a535-83ef2d458d67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:02.026891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-73bde7f9-bf69-4cd5-a535-83ef2d458d67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:02.028342 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-73bde7f9-bf69-4cd5-a535-83ef2d458d67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:02.029486 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-73bde7f9-bf69-4cd5-a535-83ef2d458d67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1721 microversion: 2.20 time: 0.174086 Oct 28 09:48:02.030025 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 238/459] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:01 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1721 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:02.191988 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-ba603964-5a13-452d-b0ad-eb0dcd0d3324 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:02.192529 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-ba603964-5a13-452d-b0ad-eb0dcd0d3324 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:02.198759 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ba603964-5a13-452d-b0ad-eb0dcd0d3324 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:02.199234 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ba603964-5a13-452d-b0ad-eb0dcd0d3324 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:02.199659 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ba603964-5a13-452d-b0ad-eb0dcd0d3324 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:02.200334 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ba603964-5a13-452d-b0ad-eb0dcd0d3324 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:02.200774 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ba603964-5a13-452d-b0ad-eb0dcd0d3324 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:02.201187 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ba603964-5a13-452d-b0ad-eb0dcd0d3324 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:02.340460 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-ba603964-5a13-452d-b0ad-eb0dcd0d3324 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:02.348344 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-ba603964-5a13-452d-b0ad-eb0dcd0d3324 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1557 microversion: 2.1 time: 0.160807 Oct 28 09:48:02.349712 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 239/460] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:02 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1557 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:48:02.371214 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-90be8d7c-7c38-45f4-9ccb-c7a999b2587a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:02.371960 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-90be8d7c-7c38-45f4-9ccb-c7a999b2587a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:02.379526 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-90be8d7c-7c38-45f4-9ccb-c7a999b2587a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:02.380199 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-90be8d7c-7c38-45f4-9ccb-c7a999b2587a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:02.380946 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-90be8d7c-7c38-45f4-9ccb-c7a999b2587a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:02.381797 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-90be8d7c-7c38-45f4-9ccb-c7a999b2587a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:02.382425 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-90be8d7c-7c38-45f4-9ccb-c7a999b2587a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:02.382868 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-90be8d7c-7c38-45f4-9ccb-c7a999b2587a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:02.439633 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7aefac91-0d9d-4613-ab43-f51acd4d33a0 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "POST /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.823980 Oct 28 09:48:02.448227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 222/461] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:47:59 2022] POST /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27/os-volume_attachments => generated 194 bytes in 2833 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:48:02.455326 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-34a91749-7964-4fa5-8e7e-3ae32eb56aed req-e0798dd9-69b0-4983-a695-16829df09382 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c716d5fd-f257-4f6a-8f52-18ad59e5c31e", "name": "network-vif-unplugged", "status": "completed", "tag": "c2310b44-82bb-408f-85b5-8a5556b92e25"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:48:02.466239 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-34a91749-7964-4fa5-8e7e-3ae32eb56aed req-e0798dd9-69b0-4983-a695-16829df09382 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:02.467260 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-34a91749-7964-4fa5-8e7e-3ae32eb56aed req-e0798dd9-69b0-4983-a695-16829df09382 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:02.467911 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-34a91749-7964-4fa5-8e7e-3ae32eb56aed req-e0798dd9-69b0-4983-a695-16829df09382 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:02.470089 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-90be8d7c-7c38-45f4-9ccb-c7a999b2587a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.102657 Oct 28 09:48:02.478628 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 240/462] 10.222.0.25 () {58 vars in 1275 bytes} [Fri Oct 28 09:48:02 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b/os-interface => generated 284 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:48:02.480880 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-34a91749-7964-4fa5-8e7e-3ae32eb56aed req-e0798dd9-69b0-4983-a695-16829df09382 service nova] Creating event network-vif-unplugged:c2310b44-82bb-408f-85b5-8a5556b92e25 for instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e on hv-cinder-zed-3 Oct 28 09:48:02.486062 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-34a91749-7964-4fa5-8e7e-3ae32eb56aed req-e0798dd9-69b0-4983-a695-16829df09382 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035230 Oct 28 09:48:02.487006 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 223/463] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:48:02 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:48:03.057598 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8626d220-e25e-4c94-801a-fdbecfb91562 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:03.058207 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8626d220-e25e-4c94-801a-fdbecfb91562 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:03.064616 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8626d220-e25e-4c94-801a-fdbecfb91562 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:03.065102 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8626d220-e25e-4c94-801a-fdbecfb91562 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:03.066078 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8626d220-e25e-4c94-801a-fdbecfb91562 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:03.066816 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8626d220-e25e-4c94-801a-fdbecfb91562 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:03.067267 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8626d220-e25e-4c94-801a-fdbecfb91562 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:03.067690 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8626d220-e25e-4c94-801a-fdbecfb91562 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:03.225084 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-8626d220-e25e-4c94-801a-fdbecfb91562 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:03.234715 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-8626d220-e25e-4c94-801a-fdbecfb91562 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:03.236169 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-8626d220-e25e-4c94-801a-fdbecfb91562 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:03.237340 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8626d220-e25e-4c94-801a-fdbecfb91562 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1721 microversion: 2.20 time: 0.184123 Oct 28 09:48:03.237929 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 241/464] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:03 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1721 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:03.692619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-8ad713fd-f819-4418-b462-652179434fbc req-bb873f39-5039-4e44-a7c9-9a22d1177862 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f8edd976-e011-4194-ad94-1160cc01c63b", "tag": "450bfe2d-019d-4361-97e8-cd9a4452887d"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:48:03.703498 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-8ad713fd-f819-4418-b462-652179434fbc req-bb873f39-5039-4e44-a7c9-9a22d1177862 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:03.703969 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-8ad713fd-f819-4418-b462-652179434fbc req-bb873f39-5039-4e44-a7c9-9a22d1177862 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:03.704403 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-8ad713fd-f819-4418-b462-652179434fbc req-bb873f39-5039-4e44-a7c9-9a22d1177862 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:03.717115 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-8ad713fd-f819-4418-b462-652179434fbc req-bb873f39-5039-4e44-a7c9-9a22d1177862 service nova] Creating event network-changed:450bfe2d-019d-4361-97e8-cd9a4452887d for instance f8edd976-e011-4194-ad94-1160cc01c63b on hv-cinder-zed-3 Oct 28 09:48:03.723095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-8ad713fd-f819-4418-b462-652179434fbc req-bb873f39-5039-4e44-a7c9-9a22d1177862 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035048 Oct 28 09:48:03.723717 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 224/465] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:48:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:48:04.267769 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e8809d63-7b51-4dca-bb4f-0f8da4f982b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:04.268375 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e8809d63-7b51-4dca-bb4f-0f8da4f982b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:04.275380 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e8809d63-7b51-4dca-bb4f-0f8da4f982b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:04.275856 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e8809d63-7b51-4dca-bb4f-0f8da4f982b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:04.276289 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e8809d63-7b51-4dca-bb4f-0f8da4f982b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:04.276972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e8809d63-7b51-4dca-bb4f-0f8da4f982b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:04.277423 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e8809d63-7b51-4dca-bb4f-0f8da4f982b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:04.277840 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e8809d63-7b51-4dca-bb4f-0f8da4f982b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:04.414162 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e8809d63-7b51-4dca-bb4f-0f8da4f982b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:04.422533 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e8809d63-7b51-4dca-bb4f-0f8da4f982b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:04.423975 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e8809d63-7b51-4dca-bb4f-0f8da4f982b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:04.425159 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e8809d63-7b51-4dca-bb4f-0f8da4f982b4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1721 microversion: 2.20 time: 0.161289 Oct 28 09:48:04.425719 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 242/466] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:04 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1721 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:05.455110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7d1bd543-6bf5-4d4e-8f40-ab554c88f71d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:05.455802 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7d1bd543-6bf5-4d4e-8f40-ab554c88f71d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:05.463509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7d1bd543-6bf5-4d4e-8f40-ab554c88f71d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:05.463993 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7d1bd543-6bf5-4d4e-8f40-ab554c88f71d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:05.464424 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7d1bd543-6bf5-4d4e-8f40-ab554c88f71d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:05.465120 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7d1bd543-6bf5-4d4e-8f40-ab554c88f71d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:05.465855 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7d1bd543-6bf5-4d4e-8f40-ab554c88f71d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:05.466275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7d1bd543-6bf5-4d4e-8f40-ab554c88f71d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:05.799421 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7d1bd543-6bf5-4d4e-8f40-ab554c88f71d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:05.808753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7d1bd543-6bf5-4d4e-8f40-ab554c88f71d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:05.810268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7d1bd543-6bf5-4d4e-8f40-ab554c88f71d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:05.811546 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7d1bd543-6bf5-4d4e-8f40-ab554c88f71d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1724 microversion: 2.20 time: 0.361197 Oct 28 09:48:05.812353 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 225/467] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:05 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1724 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:05.840038 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8e87901a-9112-4cd4-9258-8386bba5a837 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a18cc270-0f5e-4a69-9502-54ad13534acb"}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:48:05.843030 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8e87901a-9112-4cd4-9258-8386bba5a837 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:05.852545 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8e87901a-9112-4cd4-9258-8386bba5a837 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:05.853131 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8e87901a-9112-4cd4-9258-8386bba5a837 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:05.853626 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8e87901a-9112-4cd4-9258-8386bba5a837 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:05.854413 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8e87901a-9112-4cd4-9258-8386bba5a837 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:05.855010 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8e87901a-9112-4cd4-9258-8386bba5a837 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:05.855435 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8e87901a-9112-4cd4-9258-8386bba5a837 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:05.936772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:06.174369 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8e87901a-9112-4cd4-9258-8386bba5a837 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "POST /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.338197 Oct 28 09:48:06.175213 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 243/468] 10.222.0.25 () {62 vars in 1364 bytes} [Fri Oct 28 09:48:05 2022] POST /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments => generated 188 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:48:06.305160 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1c6bb363-4aae-4edf-a617-fc45fa427ded tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:48:06.305738 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1c6bb363-4aae-4edf-a617-fc45fa427ded tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:06.312298 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1c6bb363-4aae-4edf-a617-fc45fa427ded tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:06.312770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1c6bb363-4aae-4edf-a617-fc45fa427ded tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:06.313230 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1c6bb363-4aae-4edf-a617-fc45fa427ded tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:06.313905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1c6bb363-4aae-4edf-a617-fc45fa427ded tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:06.314344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1c6bb363-4aae-4edf-a617-fc45fa427ded tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:06.314778 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1c6bb363-4aae-4edf-a617-fc45fa427ded tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:06.356543 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1c6bb363-4aae-4edf-a617-fc45fa427ded tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=127030) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 28 09:48:06.428214 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1c6bb363-4aae-4edf-a617-fc45fa427ded tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "POST /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/action" status: 202 len: 0 microversion: 2.20 time: 0.126755 Oct 28 09:48:06.428731 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 226/469] 10.222.0.25 () {62 vars in 1319 bytes} [Fri Oct 28 09:48:06 2022] POST /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/action => generated 0 bytes in 129 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 28 09:48:06.437275 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-27a9a189-4ca3-4d2e-b0f7-a3b8b7b05dd6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:06.438225 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-27a9a189-4ca3-4d2e-b0f7-a3b8b7b05dd6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:06.446750 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-27a9a189-4ca3-4d2e-b0f7-a3b8b7b05dd6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:06.447549 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-27a9a189-4ca3-4d2e-b0f7-a3b8b7b05dd6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:06.448286 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-27a9a189-4ca3-4d2e-b0f7-a3b8b7b05dd6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:06.449265 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-27a9a189-4ca3-4d2e-b0f7-a3b8b7b05dd6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:06.449911 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-27a9a189-4ca3-4d2e-b0f7-a3b8b7b05dd6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:06.450346 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-27a9a189-4ca3-4d2e-b0f7-a3b8b7b05dd6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:06.588064 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-27a9a189-4ca3-4d2e-b0f7-a3b8b7b05dd6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:06.597455 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-27a9a189-4ca3-4d2e-b0f7-a3b8b7b05dd6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:06.598939 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-27a9a189-4ca3-4d2e-b0f7-a3b8b7b05dd6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:06.600096 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-27a9a189-4ca3-4d2e-b0f7-a3b8b7b05dd6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1810 microversion: 2.20 time: 0.167017 Oct 28 09:48:06.600642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 244/470] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:06 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1810 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:07.626109 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-02da7ab2-053c-4c2e-829f-fd64d65e4d97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:07.626973 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-02da7ab2-053c-4c2e-829f-fd64d65e4d97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:07.633302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-02da7ab2-053c-4c2e-829f-fd64d65e4d97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:07.633778 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-02da7ab2-053c-4c2e-829f-fd64d65e4d97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:07.634201 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-02da7ab2-053c-4c2e-829f-fd64d65e4d97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:07.634907 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-02da7ab2-053c-4c2e-829f-fd64d65e4d97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:07.635350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-02da7ab2-053c-4c2e-829f-fd64d65e4d97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:07.635758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-02da7ab2-053c-4c2e-829f-fd64d65e4d97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:07.768349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-02da7ab2-053c-4c2e-829f-fd64d65e4d97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:07.776968 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-02da7ab2-053c-4c2e-829f-fd64d65e4d97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:07.778503 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-02da7ab2-053c-4c2e-829f-fd64d65e4d97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:07.779650 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-02da7ab2-053c-4c2e-829f-fd64d65e4d97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1752 microversion: 2.20 time: 0.158635 Oct 28 09:48:07.780204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 227/471] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:07 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1752 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:08.805529 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5f873ff7-bb67-40fe-aa70-9d9b358321df tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:08.806131 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5f873ff7-bb67-40fe-aa70-9d9b358321df tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:08.812613 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5f873ff7-bb67-40fe-aa70-9d9b358321df tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:08.813168 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5f873ff7-bb67-40fe-aa70-9d9b358321df tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:08.813602 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5f873ff7-bb67-40fe-aa70-9d9b358321df tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:08.814282 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5f873ff7-bb67-40fe-aa70-9d9b358321df tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:08.814760 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5f873ff7-bb67-40fe-aa70-9d9b358321df tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:08.815179 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5f873ff7-bb67-40fe-aa70-9d9b358321df tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:08.944825 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5f873ff7-bb67-40fe-aa70-9d9b358321df tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:08.953319 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5f873ff7-bb67-40fe-aa70-9d9b358321df tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:08.954850 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-5f873ff7-bb67-40fe-aa70-9d9b358321df tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:08.956056 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5f873ff7-bb67-40fe-aa70-9d9b358321df tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1752 microversion: 2.20 time: 0.154358 Oct 28 09:48:08.957107 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 245/472] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:08 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1752 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:09.982675 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-cae7ef25-7a51-4f7a-9120-58f198824b6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:09.983390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-cae7ef25-7a51-4f7a-9120-58f198824b6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:09.991534 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cae7ef25-7a51-4f7a-9120-58f198824b6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:09.992008 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cae7ef25-7a51-4f7a-9120-58f198824b6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:09.992537 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cae7ef25-7a51-4f7a-9120-58f198824b6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:09.993213 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cae7ef25-7a51-4f7a-9120-58f198824b6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:09.993657 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cae7ef25-7a51-4f7a-9120-58f198824b6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:09.994068 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cae7ef25-7a51-4f7a-9120-58f198824b6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:10.132950 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-cae7ef25-7a51-4f7a-9120-58f198824b6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:10.142051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-cae7ef25-7a51-4f7a-9120-58f198824b6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:10.144049 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-cae7ef25-7a51-4f7a-9120-58f198824b6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:10.145201 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-cae7ef25-7a51-4f7a-9120-58f198824b6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1752 microversion: 2.20 time: 0.167454 Oct 28 09:48:10.145764 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 228/473] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:09 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1752 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:11.170343 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-bb57d2b6-4b65-415d-9da9-fee2005012ec tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:11.171005 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-bb57d2b6-4b65-415d-9da9-fee2005012ec tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:11.177980 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bb57d2b6-4b65-415d-9da9-fee2005012ec tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:11.178479 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bb57d2b6-4b65-415d-9da9-fee2005012ec tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:11.178906 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bb57d2b6-4b65-415d-9da9-fee2005012ec tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:11.179580 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bb57d2b6-4b65-415d-9da9-fee2005012ec tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:11.180018 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bb57d2b6-4b65-415d-9da9-fee2005012ec tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:11.180429 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bb57d2b6-4b65-415d-9da9-fee2005012ec tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:11.317505 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-bb57d2b6-4b65-415d-9da9-fee2005012ec tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:11.326258 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-bb57d2b6-4b65-415d-9da9-fee2005012ec tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:11.327730 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-bb57d2b6-4b65-415d-9da9-fee2005012ec tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:11.328957 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-bb57d2b6-4b65-415d-9da9-fee2005012ec tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1752 microversion: 2.20 time: 0.162730 Oct 28 09:48:11.329561 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 246/474] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:11 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1752 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:12.364745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6b7698a7-93b7-479f-a50a-9318daefd8eb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:12.366031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6b7698a7-93b7-479f-a50a-9318daefd8eb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:12.374308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b7698a7-93b7-479f-a50a-9318daefd8eb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:12.374993 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b7698a7-93b7-479f-a50a-9318daefd8eb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:12.375490 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b7698a7-93b7-479f-a50a-9318daefd8eb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:12.376351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b7698a7-93b7-479f-a50a-9318daefd8eb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:12.376866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b7698a7-93b7-479f-a50a-9318daefd8eb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:12.377365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b7698a7-93b7-479f-a50a-9318daefd8eb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:12.512035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6b7698a7-93b7-479f-a50a-9318daefd8eb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:12.521825 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6b7698a7-93b7-479f-a50a-9318daefd8eb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:12.524107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6b7698a7-93b7-479f-a50a-9318daefd8eb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:12.525651 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6b7698a7-93b7-479f-a50a-9318daefd8eb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1752 microversion: 2.20 time: 0.167688 Oct 28 09:48:12.526209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 229/475] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:12 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1752 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:13.551887 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-83cf126a-ba56-4f91-b411-ba6dd4932544 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:13.552607 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-83cf126a-ba56-4f91-b411-ba6dd4932544 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:13.555081 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=72 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:13.560636 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-83cf126a-ba56-4f91-b411-ba6dd4932544 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:13.561119 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-83cf126a-ba56-4f91-b411-ba6dd4932544 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:13.561550 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-83cf126a-ba56-4f91-b411-ba6dd4932544 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:13.562248 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-83cf126a-ba56-4f91-b411-ba6dd4932544 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:13.562765 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-83cf126a-ba56-4f91-b411-ba6dd4932544 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:13.563185 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-83cf126a-ba56-4f91-b411-ba6dd4932544 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:13.695970 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-83cf126a-ba56-4f91-b411-ba6dd4932544 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:13.704857 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-83cf126a-ba56-4f91-b411-ba6dd4932544 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:13.706336 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-83cf126a-ba56-4f91-b411-ba6dd4932544 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:13.707502 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-83cf126a-ba56-4f91-b411-ba6dd4932544 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.159560 Oct 28 09:48:13.708242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 247/476] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:13 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:14.733755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ae07719a-b738-4bba-884d-967b7d8ff528 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:14.734274 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ae07719a-b738-4bba-884d-967b7d8ff528 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:14.740815 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae07719a-b738-4bba-884d-967b7d8ff528 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:14.741290 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae07719a-b738-4bba-884d-967b7d8ff528 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:14.741717 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae07719a-b738-4bba-884d-967b7d8ff528 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:14.742425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae07719a-b738-4bba-884d-967b7d8ff528 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:14.742863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae07719a-b738-4bba-884d-967b7d8ff528 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:14.743279 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae07719a-b738-4bba-884d-967b7d8ff528 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:14.871608 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ae07719a-b738-4bba-884d-967b7d8ff528 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:14.880661 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ae07719a-b738-4bba-884d-967b7d8ff528 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:14.882078 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ae07719a-b738-4bba-884d-967b7d8ff528 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:14.883252 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ae07719a-b738-4bba-884d-967b7d8ff528 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.153892 Oct 28 09:48:14.883811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 230/477] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:14 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:15.910317 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-9845e5ee-77d8-4081-8a12-068cd77dd147 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:15.910949 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-9845e5ee-77d8-4081-8a12-068cd77dd147 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:15.917420 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9845e5ee-77d8-4081-8a12-068cd77dd147 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:15.918150 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9845e5ee-77d8-4081-8a12-068cd77dd147 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:15.918615 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9845e5ee-77d8-4081-8a12-068cd77dd147 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:15.919292 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9845e5ee-77d8-4081-8a12-068cd77dd147 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:15.919741 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9845e5ee-77d8-4081-8a12-068cd77dd147 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:15.920153 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9845e5ee-77d8-4081-8a12-068cd77dd147 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:16.053425 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-9845e5ee-77d8-4081-8a12-068cd77dd147 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:16.062112 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-9845e5ee-77d8-4081-8a12-068cd77dd147 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:16.063581 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-9845e5ee-77d8-4081-8a12-068cd77dd147 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:16.064715 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-9845e5ee-77d8-4081-8a12-068cd77dd147 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.158941 Oct 28 09:48:16.065290 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 248/478] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:15 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:17.090320 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ecb0ed14-0e08-4dc5-a3e8-1ffde8d15bc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:17.090945 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ecb0ed14-0e08-4dc5-a3e8-1ffde8d15bc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:17.093444 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:INSERT=3,nova_api:SELECT=71,nova_api:UPDATE=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:17.099720 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ecb0ed14-0e08-4dc5-a3e8-1ffde8d15bc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:17.100198 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ecb0ed14-0e08-4dc5-a3e8-1ffde8d15bc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:17.100623 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ecb0ed14-0e08-4dc5-a3e8-1ffde8d15bc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:17.101305 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ecb0ed14-0e08-4dc5-a3e8-1ffde8d15bc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:17.101749 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ecb0ed14-0e08-4dc5-a3e8-1ffde8d15bc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:17.102144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ecb0ed14-0e08-4dc5-a3e8-1ffde8d15bc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:17.242498 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ecb0ed14-0e08-4dc5-a3e8-1ffde8d15bc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:17.251696 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ecb0ed14-0e08-4dc5-a3e8-1ffde8d15bc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:17.253279 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ecb0ed14-0e08-4dc5-a3e8-1ffde8d15bc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:17.254662 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ecb0ed14-0e08-4dc5-a3e8-1ffde8d15bc7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.168199 Oct 28 09:48:17.255263 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 231/479] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:17 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:17.934935 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-cc913177-d230-4f41-88d3-3b2acd9e6ae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:48:17.937148 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-cc913177-d230-4f41-88d3-3b2acd9e6ae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:17.944019 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cc913177-d230-4f41-88d3-3b2acd9e6ae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:17.944508 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cc913177-d230-4f41-88d3-3b2acd9e6ae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:17.944932 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cc913177-d230-4f41-88d3-3b2acd9e6ae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:17.945612 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cc913177-d230-4f41-88d3-3b2acd9e6ae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:17.946063 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cc913177-d230-4f41-88d3-3b2acd9e6ae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:17.946508 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cc913177-d230-4f41-88d3-3b2acd9e6ae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:18.017063 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-cc913177-d230-4f41-88d3-3b2acd9e6ae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "POST /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/action" status: 202 len: 0 microversion: 2.1 time: 0.085878 Oct 28 09:48:18.021219 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 249/480] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:48:17 2022] POST /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/action => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:48:18.026432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b1a296ea-606c-4243-856f-2e647ac643bd tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:18.027044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b1a296ea-606c-4243-856f-2e647ac643bd tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:18.033225 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b1a296ea-606c-4243-856f-2e647ac643bd tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:18.033701 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b1a296ea-606c-4243-856f-2e647ac643bd tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:18.034684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b1a296ea-606c-4243-856f-2e647ac643bd tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:18.035361 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b1a296ea-606c-4243-856f-2e647ac643bd tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:18.035808 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b1a296ea-606c-4243-856f-2e647ac643bd tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:18.036224 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b1a296ea-606c-4243-856f-2e647ac643bd tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:18.177722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b1a296ea-606c-4243-856f-2e647ac643bd tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:18.185921 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b1a296ea-606c-4243-856f-2e647ac643bd tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.163365 Oct 28 09:48:18.186442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 232/481] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:18 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:18.283799 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2b57162a-56c2-419d-a2eb-9117664f6314 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:18.284993 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2b57162a-56c2-419d-a2eb-9117664f6314 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:18.291132 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2b57162a-56c2-419d-a2eb-9117664f6314 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:18.291608 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2b57162a-56c2-419d-a2eb-9117664f6314 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:18.292043 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2b57162a-56c2-419d-a2eb-9117664f6314 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:18.292719 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2b57162a-56c2-419d-a2eb-9117664f6314 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:18.293434 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2b57162a-56c2-419d-a2eb-9117664f6314 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:18.293839 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2b57162a-56c2-419d-a2eb-9117664f6314 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:18.431355 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2b57162a-56c2-419d-a2eb-9117664f6314 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:18.439851 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2b57162a-56c2-419d-a2eb-9117664f6314 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:18.441206 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2b57162a-56c2-419d-a2eb-9117664f6314 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:18.442344 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2b57162a-56c2-419d-a2eb-9117664f6314 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.162803 Oct 28 09:48:18.442918 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 250/482] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:18 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:19.209473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-996c8d72-bc91-42e5-bc2f-2834c23c2148 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:19.210083 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-996c8d72-bc91-42e5-bc2f-2834c23c2148 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:19.216448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-996c8d72-bc91-42e5-bc2f-2834c23c2148 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:19.216917 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-996c8d72-bc91-42e5-bc2f-2834c23c2148 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:19.217344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-996c8d72-bc91-42e5-bc2f-2834c23c2148 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:19.218016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-996c8d72-bc91-42e5-bc2f-2834c23c2148 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:19.218486 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-996c8d72-bc91-42e5-bc2f-2834c23c2148 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:19.218900 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-996c8d72-bc91-42e5-bc2f-2834c23c2148 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:19.358131 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-996c8d72-bc91-42e5-bc2f-2834c23c2148 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:19.366337 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-996c8d72-bc91-42e5-bc2f-2834c23c2148 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.161191 Oct 28 09:48:19.366940 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 233/483] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:19 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:19.468106 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-aafcb7b6-7695-4dd3-ad0c-067c6868da7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:19.468624 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-aafcb7b6-7695-4dd3-ad0c-067c6868da7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:19.475110 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aafcb7b6-7695-4dd3-ad0c-067c6868da7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:19.475581 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aafcb7b6-7695-4dd3-ad0c-067c6868da7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:19.476014 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aafcb7b6-7695-4dd3-ad0c-067c6868da7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:19.476730 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aafcb7b6-7695-4dd3-ad0c-067c6868da7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:19.477199 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aafcb7b6-7695-4dd3-ad0c-067c6868da7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:19.477588 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-aafcb7b6-7695-4dd3-ad0c-067c6868da7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:19.609764 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-aafcb7b6-7695-4dd3-ad0c-067c6868da7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:19.618413 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-aafcb7b6-7695-4dd3-ad0c-067c6868da7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:19.619826 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-aafcb7b6-7695-4dd3-ad0c-067c6868da7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:19.620992 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-aafcb7b6-7695-4dd3-ad0c-067c6868da7b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.156843 Oct 28 09:48:19.621544 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 251/484] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:19 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:20.389148 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4600d8c0-5675-4f14-9551-6c1cb643b078 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:20.389656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4600d8c0-5675-4f14-9551-6c1cb643b078 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:20.396340 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4600d8c0-5675-4f14-9551-6c1cb643b078 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:20.396820 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4600d8c0-5675-4f14-9551-6c1cb643b078 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:20.397249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4600d8c0-5675-4f14-9551-6c1cb643b078 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:20.397937 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4600d8c0-5675-4f14-9551-6c1cb643b078 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:20.398449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4600d8c0-5675-4f14-9551-6c1cb643b078 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:20.398876 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4600d8c0-5675-4f14-9551-6c1cb643b078 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:20.587170 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-4600d8c0-5675-4f14-9551-6c1cb643b078 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:20.596514 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4600d8c0-5675-4f14-9551-6c1cb643b078 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.211132 Oct 28 09:48:20.597102 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 234/485] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:20 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:20.647397 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b144831c-c47d-430b-bc1b-12bf506900f2 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:20.647987 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b144831c-c47d-430b-bc1b-12bf506900f2 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:20.655380 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b144831c-c47d-430b-bc1b-12bf506900f2 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:20.659840 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b144831c-c47d-430b-bc1b-12bf506900f2 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:20.660629 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b144831c-c47d-430b-bc1b-12bf506900f2 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:20.661746 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b144831c-c47d-430b-bc1b-12bf506900f2 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:20.662371 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b144831c-c47d-430b-bc1b-12bf506900f2 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:20.665120 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b144831c-c47d-430b-bc1b-12bf506900f2 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:20.835230 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b144831c-c47d-430b-bc1b-12bf506900f2 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:20.844289 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b144831c-c47d-430b-bc1b-12bf506900f2 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:20.845739 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b144831c-c47d-430b-bc1b-12bf506900f2 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:20.847472 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b144831c-c47d-430b-bc1b-12bf506900f2 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.204451 Oct 28 09:48:20.862724 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 252/486] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:20 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:21.620769 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-48b0d7b5-0834-4943-9e06-92aedbe5cae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:21.621382 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-48b0d7b5-0834-4943-9e06-92aedbe5cae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:21.628421 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-48b0d7b5-0834-4943-9e06-92aedbe5cae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:21.628989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-48b0d7b5-0834-4943-9e06-92aedbe5cae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:21.629426 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-48b0d7b5-0834-4943-9e06-92aedbe5cae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:21.630131 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-48b0d7b5-0834-4943-9e06-92aedbe5cae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:21.630607 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-48b0d7b5-0834-4943-9e06-92aedbe5cae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:21.631032 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-48b0d7b5-0834-4943-9e06-92aedbe5cae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:21.798966 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-48b0d7b5-0834-4943-9e06-92aedbe5cae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:21.809918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-48b0d7b5-0834-4943-9e06-92aedbe5cae7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.192869 Oct 28 09:48:21.810519 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 235/487] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:21 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:21.875463 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-94a440c1-788e-4ac1-a3cf-567dd2ff4236 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:21.876069 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-94a440c1-788e-4ac1-a3cf-567dd2ff4236 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:21.882717 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94a440c1-788e-4ac1-a3cf-567dd2ff4236 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:21.883191 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94a440c1-788e-4ac1-a3cf-567dd2ff4236 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:21.883678 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94a440c1-788e-4ac1-a3cf-567dd2ff4236 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:21.884363 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94a440c1-788e-4ac1-a3cf-567dd2ff4236 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:21.884809 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94a440c1-788e-4ac1-a3cf-567dd2ff4236 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:21.885228 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94a440c1-788e-4ac1-a3cf-567dd2ff4236 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:22.041506 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-94a440c1-788e-4ac1-a3cf-567dd2ff4236 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:22.051911 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-94a440c1-788e-4ac1-a3cf-567dd2ff4236 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:22.053349 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-94a440c1-788e-4ac1-a3cf-567dd2ff4236 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:22.054550 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-94a440c1-788e-4ac1-a3cf-567dd2ff4236 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.183096 Oct 28 09:48:22.055287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 253/488] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:21 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:22.833469 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c35d5249-09e6-446b-b2a0-50c34a52e17d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:22.834115 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c35d5249-09e6-446b-b2a0-50c34a52e17d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:22.841067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c35d5249-09e6-446b-b2a0-50c34a52e17d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:22.841657 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c35d5249-09e6-446b-b2a0-50c34a52e17d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:22.842007 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c35d5249-09e6-446b-b2a0-50c34a52e17d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:22.842732 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c35d5249-09e6-446b-b2a0-50c34a52e17d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:22.843193 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c35d5249-09e6-446b-b2a0-50c34a52e17d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:22.843596 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c35d5249-09e6-446b-b2a0-50c34a52e17d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:22.847919 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:22.998683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c35d5249-09e6-446b-b2a0-50c34a52e17d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:23.011647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c35d5249-09e6-446b-b2a0-50c34a52e17d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.182054 Oct 28 09:48:23.021733 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 236/489] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:22 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:23.084405 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-12ac4ee8-8d44-4321-84e5-2fcca1bc8069 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:23.085115 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-12ac4ee8-8d44-4321-84e5-2fcca1bc8069 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:23.092112 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-12ac4ee8-8d44-4321-84e5-2fcca1bc8069 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:23.092854 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-12ac4ee8-8d44-4321-84e5-2fcca1bc8069 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:23.093457 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-12ac4ee8-8d44-4321-84e5-2fcca1bc8069 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:23.095202 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-12ac4ee8-8d44-4321-84e5-2fcca1bc8069 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:23.095822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-12ac4ee8-8d44-4321-84e5-2fcca1bc8069 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:23.096450 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-12ac4ee8-8d44-4321-84e5-2fcca1bc8069 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:23.248891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-12ac4ee8-8d44-4321-84e5-2fcca1bc8069 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:23.259134 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-12ac4ee8-8d44-4321-84e5-2fcca1bc8069 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:23.260581 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-12ac4ee8-8d44-4321-84e5-2fcca1bc8069 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:23.261826 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-12ac4ee8-8d44-4321-84e5-2fcca1bc8069 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.182194 Oct 28 09:48:23.262474 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 254/490] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:23 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:24.035400 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d802ce79-1aab-4cfe-8c67-98656e4f2018 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:24.036002 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d802ce79-1aab-4cfe-8c67-98656e4f2018 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:24.042821 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d802ce79-1aab-4cfe-8c67-98656e4f2018 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:24.043562 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d802ce79-1aab-4cfe-8c67-98656e4f2018 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:24.044078 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d802ce79-1aab-4cfe-8c67-98656e4f2018 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:24.044885 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d802ce79-1aab-4cfe-8c67-98656e4f2018 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:24.045354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d802ce79-1aab-4cfe-8c67-98656e4f2018 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:24.045747 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d802ce79-1aab-4cfe-8c67-98656e4f2018 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:24.215026 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d802ce79-1aab-4cfe-8c67-98656e4f2018 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:24.224037 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d802ce79-1aab-4cfe-8c67-98656e4f2018 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.193200 Oct 28 09:48:24.224587 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 237/491] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:24 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:24.290113 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-132d7055-04c3-4425-9e23-64a081a6d190 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:24.291540 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-132d7055-04c3-4425-9e23-64a081a6d190 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:24.299147 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-132d7055-04c3-4425-9e23-64a081a6d190 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:24.299770 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-132d7055-04c3-4425-9e23-64a081a6d190 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:24.300279 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-132d7055-04c3-4425-9e23-64a081a6d190 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:24.301903 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-132d7055-04c3-4425-9e23-64a081a6d190 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:24.301903 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-132d7055-04c3-4425-9e23-64a081a6d190 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:24.302146 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-132d7055-04c3-4425-9e23-64a081a6d190 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:24.449378 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-132d7055-04c3-4425-9e23-64a081a6d190 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:24.459469 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-132d7055-04c3-4425-9e23-64a081a6d190 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:24.460891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-132d7055-04c3-4425-9e23-64a081a6d190 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:24.462039 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-132d7055-04c3-4425-9e23-64a081a6d190 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.177058 Oct 28 09:48:24.462613 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 255/492] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:24 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:25.247452 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-75a01bf2-c5c0-4fe9-99fc-dc4abefeb519 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:25.248213 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-75a01bf2-c5c0-4fe9-99fc-dc4abefeb519 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:25.255209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-75a01bf2-c5c0-4fe9-99fc-dc4abefeb519 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:25.255665 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-75a01bf2-c5c0-4fe9-99fc-dc4abefeb519 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:25.256092 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-75a01bf2-c5c0-4fe9-99fc-dc4abefeb519 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:25.256769 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-75a01bf2-c5c0-4fe9-99fc-dc4abefeb519 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:25.257215 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-75a01bf2-c5c0-4fe9-99fc-dc4abefeb519 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:25.257628 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-75a01bf2-c5c0-4fe9-99fc-dc4abefeb519 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:25.425548 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-75a01bf2-c5c0-4fe9-99fc-dc4abefeb519 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:25.435979 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-75a01bf2-c5c0-4fe9-99fc-dc4abefeb519 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.192292 Oct 28 09:48:25.436555 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 238/493] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:25 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:25.495401 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b95194cd-bf1d-42ec-a5c5-25e90a02123f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:25.496168 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b95194cd-bf1d-42ec-a5c5-25e90a02123f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:25.502682 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b95194cd-bf1d-42ec-a5c5-25e90a02123f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:25.503170 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b95194cd-bf1d-42ec-a5c5-25e90a02123f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:25.503605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b95194cd-bf1d-42ec-a5c5-25e90a02123f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:25.504328 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b95194cd-bf1d-42ec-a5c5-25e90a02123f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:25.504946 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b95194cd-bf1d-42ec-a5c5-25e90a02123f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:25.505523 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b95194cd-bf1d-42ec-a5c5-25e90a02123f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:25.660362 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b95194cd-bf1d-42ec-a5c5-25e90a02123f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:25.664376 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:25.670868 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b95194cd-bf1d-42ec-a5c5-25e90a02123f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:25.672296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b95194cd-bf1d-42ec-a5c5-25e90a02123f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:25.673445 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b95194cd-bf1d-42ec-a5c5-25e90a02123f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.182409 Oct 28 09:48:25.674000 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 256/494] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:25 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:26.460499 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-45783da3-24f4-4e7b-a046-473105663ded tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:26.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-45783da3-24f4-4e7b-a046-473105663ded tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:26.467526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-45783da3-24f4-4e7b-a046-473105663ded tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:26.468002 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-45783da3-24f4-4e7b-a046-473105663ded tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:26.468427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-45783da3-24f4-4e7b-a046-473105663ded tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:26.469102 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-45783da3-24f4-4e7b-a046-473105663ded tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:26.469884 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-45783da3-24f4-4e7b-a046-473105663ded tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:26.470244 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-45783da3-24f4-4e7b-a046-473105663ded tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:26.678780 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-45783da3-24f4-4e7b-a046-473105663ded tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:26.688645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-45783da3-24f4-4e7b-a046-473105663ded tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.231995 Oct 28 09:48:26.698973 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 239/495] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:26 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:26.707399 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d299e34f-44ab-47d9-89fc-d2b2461309f5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:26.708116 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d299e34f-44ab-47d9-89fc-d2b2461309f5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:26.716359 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d299e34f-44ab-47d9-89fc-d2b2461309f5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:26.717515 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d299e34f-44ab-47d9-89fc-d2b2461309f5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:26.718251 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d299e34f-44ab-47d9-89fc-d2b2461309f5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:26.720176 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d299e34f-44ab-47d9-89fc-d2b2461309f5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:26.720960 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d299e34f-44ab-47d9-89fc-d2b2461309f5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:26.722245 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d299e34f-44ab-47d9-89fc-d2b2461309f5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:26.884188 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d299e34f-44ab-47d9-89fc-d2b2461309f5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:26.903437 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d299e34f-44ab-47d9-89fc-d2b2461309f5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:26.904856 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d299e34f-44ab-47d9-89fc-d2b2461309f5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:26.906242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d299e34f-44ab-47d9-89fc-d2b2461309f5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.205705 Oct 28 09:48:26.907199 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 257/496] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:26 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:27.714319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5e500270-263e-4cdb-ab57-509916acf493 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:27.716739 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5e500270-263e-4cdb-ab57-509916acf493 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:27.725024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e500270-263e-4cdb-ab57-509916acf493 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:27.725629 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e500270-263e-4cdb-ab57-509916acf493 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:27.726321 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e500270-263e-4cdb-ab57-509916acf493 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:27.727117 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e500270-263e-4cdb-ab57-509916acf493 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:27.727574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e500270-263e-4cdb-ab57-509916acf493 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:27.727996 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e500270-263e-4cdb-ab57-509916acf493 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:27.893264 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5e500270-263e-4cdb-ab57-509916acf493 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:27.902412 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5e500270-263e-4cdb-ab57-509916acf493 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.192627 Oct 28 09:48:27.903315 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 240/497] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:27 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:27.933583 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-738e2a46-1eb6-40bb-9282-b371e0293516 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:27.934414 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-738e2a46-1eb6-40bb-9282-b371e0293516 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:27.941724 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-738e2a46-1eb6-40bb-9282-b371e0293516 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:27.942210 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-738e2a46-1eb6-40bb-9282-b371e0293516 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:27.942683 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-738e2a46-1eb6-40bb-9282-b371e0293516 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:27.943388 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-738e2a46-1eb6-40bb-9282-b371e0293516 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:27.943834 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-738e2a46-1eb6-40bb-9282-b371e0293516 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:27.944403 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-738e2a46-1eb6-40bb-9282-b371e0293516 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:28.088041 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-738e2a46-1eb6-40bb-9282-b371e0293516 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:28.101356 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-738e2a46-1eb6-40bb-9282-b371e0293516 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:28.104207 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-738e2a46-1eb6-40bb-9282-b371e0293516 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:28.105659 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-738e2a46-1eb6-40bb-9282-b371e0293516 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.176338 Oct 28 09:48:28.106719 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 258/498] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:27 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:28.493875 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d6b63208-04ef-49ef-a01a-ac8fa418d51d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:28.494493 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d6b63208-04ef-49ef-a01a-ac8fa418d51d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:28.500741 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6b63208-04ef-49ef-a01a-ac8fa418d51d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:28.501303 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6b63208-04ef-49ef-a01a-ac8fa418d51d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:28.501802 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6b63208-04ef-49ef-a01a-ac8fa418d51d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:28.503254 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6b63208-04ef-49ef-a01a-ac8fa418d51d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:28.503902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6b63208-04ef-49ef-a01a-ac8fa418d51d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:28.504393 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d6b63208-04ef-49ef-a01a-ac8fa418d51d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:28.636971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-d6b63208-04ef-49ef-a01a-ac8fa418d51d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lazy-loading 'services' on Instance uuid a50a18b7-b11e-48cf-b8dc-58d39afc4d27 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:48:28.763421 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d6b63208-04ef-49ef-a01a-ac8fa418d51d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27/os-volume_attachments/f68b5d8c-4184-4295-98e7-4b8b535e632b" status: 202 len: 0 microversion: 2.1 time: 0.274428 Oct 28 09:48:28.764139 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 241/499] 10.222.0.25 () {58 vars in 1416 bytes} [Fri Oct 28 09:48:28 2022] DELETE /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27/os-volume_attachments/f68b5d8c-4184-4295-98e7-4b8b535e632b => generated 0 bytes in 277 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:48:28.926855 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-0ac3046d-e7ea-4ed6-8300-36c9a80ae1b4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:28.927368 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-0ac3046d-e7ea-4ed6-8300-36c9a80ae1b4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:28.933321 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ac3046d-e7ea-4ed6-8300-36c9a80ae1b4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:28.933793 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ac3046d-e7ea-4ed6-8300-36c9a80ae1b4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:28.934368 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ac3046d-e7ea-4ed6-8300-36c9a80ae1b4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:28.934912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ac3046d-e7ea-4ed6-8300-36c9a80ae1b4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:28.935356 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ac3046d-e7ea-4ed6-8300-36c9a80ae1b4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:28.935767 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0ac3046d-e7ea-4ed6-8300-36c9a80ae1b4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:29.092426 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0ac3046d-e7ea-4ed6-8300-36c9a80ae1b4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:29.100612 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-0ac3046d-e7ea-4ed6-8300-36c9a80ae1b4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.177908 Oct 28 09:48:29.101171 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 259/500] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:28 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:29.132470 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d7e4c739-0ac8-4b61-9fec-e5294e9ed6dc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:29.133255 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d7e4c739-0ac8-4b61-9fec-e5294e9ed6dc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:29.140210 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d7e4c739-0ac8-4b61-9fec-e5294e9ed6dc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:29.140701 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d7e4c739-0ac8-4b61-9fec-e5294e9ed6dc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:29.141135 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d7e4c739-0ac8-4b61-9fec-e5294e9ed6dc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:29.142020 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d7e4c739-0ac8-4b61-9fec-e5294e9ed6dc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:29.142493 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d7e4c739-0ac8-4b61-9fec-e5294e9ed6dc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:29.142916 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d7e4c739-0ac8-4b61-9fec-e5294e9ed6dc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:29.281277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d7e4c739-0ac8-4b61-9fec-e5294e9ed6dc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:29.289784 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d7e4c739-0ac8-4b61-9fec-e5294e9ed6dc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:29.291243 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d7e4c739-0ac8-4b61-9fec-e5294e9ed6dc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:29.292377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d7e4c739-0ac8-4b61-9fec-e5294e9ed6dc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.164849 Oct 28 09:48:29.292922 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 242/501] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:29 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:30.124940 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8f2ae1f8-0c32-4e50-81ab-e19d7341e3a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:30.125515 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8f2ae1f8-0c32-4e50-81ab-e19d7341e3a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:30.133095 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8f2ae1f8-0c32-4e50-81ab-e19d7341e3a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:30.133568 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8f2ae1f8-0c32-4e50-81ab-e19d7341e3a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:30.133993 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8f2ae1f8-0c32-4e50-81ab-e19d7341e3a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:30.134708 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8f2ae1f8-0c32-4e50-81ab-e19d7341e3a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:30.135155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8f2ae1f8-0c32-4e50-81ab-e19d7341e3a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:30.135570 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8f2ae1f8-0c32-4e50-81ab-e19d7341e3a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:30.292070 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-8f2ae1f8-0c32-4e50-81ab-e19d7341e3a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:30.302072 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8f2ae1f8-0c32-4e50-81ab-e19d7341e3a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.181182 Oct 28 09:48:30.302722 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 260/502] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:30 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:30.318025 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-820efc69-c5e5-4df7-8220-0d413025a0e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:30.318685 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-820efc69-c5e5-4df7-8220-0d413025a0e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:30.325093 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-820efc69-c5e5-4df7-8220-0d413025a0e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:30.325571 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-820efc69-c5e5-4df7-8220-0d413025a0e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:30.326000 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-820efc69-c5e5-4df7-8220-0d413025a0e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:30.326704 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-820efc69-c5e5-4df7-8220-0d413025a0e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:30.327145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-820efc69-c5e5-4df7-8220-0d413025a0e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:30.327562 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-820efc69-c5e5-4df7-8220-0d413025a0e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:30.463991 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-820efc69-c5e5-4df7-8220-0d413025a0e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:30.473400 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-820efc69-c5e5-4df7-8220-0d413025a0e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:30.474909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-820efc69-c5e5-4df7-8220-0d413025a0e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:30.476090 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-820efc69-c5e5-4df7-8220-0d413025a0e1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.162025 Oct 28 09:48:30.476647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 243/503] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:30 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:31.325043 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-279579f5-6af5-47c6-bfa1-07214f234251 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:31.325747 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-279579f5-6af5-47c6-bfa1-07214f234251 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:31.332782 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-279579f5-6af5-47c6-bfa1-07214f234251 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:31.333372 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-279579f5-6af5-47c6-bfa1-07214f234251 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:31.333923 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-279579f5-6af5-47c6-bfa1-07214f234251 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:31.334742 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-279579f5-6af5-47c6-bfa1-07214f234251 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:31.335260 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-279579f5-6af5-47c6-bfa1-07214f234251 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:31.335744 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-279579f5-6af5-47c6-bfa1-07214f234251 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:31.487985 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-279579f5-6af5-47c6-bfa1-07214f234251 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:31.496617 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-279579f5-6af5-47c6-bfa1-07214f234251 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.175364 Oct 28 09:48:31.497276 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 261/504] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:31 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:31.501135 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6c1d8043-3336-448c-a92a-41a930514335 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:31.501736 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6c1d8043-3336-448c-a92a-41a930514335 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:31.508547 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6c1d8043-3336-448c-a92a-41a930514335 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:31.509025 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6c1d8043-3336-448c-a92a-41a930514335 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:31.509454 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6c1d8043-3336-448c-a92a-41a930514335 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:31.510148 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6c1d8043-3336-448c-a92a-41a930514335 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:31.510693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6c1d8043-3336-448c-a92a-41a930514335 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:31.511119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6c1d8043-3336-448c-a92a-41a930514335 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:31.655941 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6c1d8043-3336-448c-a92a-41a930514335 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:31.665200 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6c1d8043-3336-448c-a92a-41a930514335 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:31.666681 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6c1d8043-3336-448c-a92a-41a930514335 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:31.667864 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6c1d8043-3336-448c-a92a-41a930514335 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.170753 Oct 28 09:48:31.668688 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 244/505] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:31 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:32.523394 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-f3fd554a-a338-42c5-a17d-106465098824 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:32.524156 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-f3fd554a-a338-42c5-a17d-106465098824 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:32.530540 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f3fd554a-a338-42c5-a17d-106465098824 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:32.531024 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f3fd554a-a338-42c5-a17d-106465098824 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:32.531455 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f3fd554a-a338-42c5-a17d-106465098824 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:32.532148 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f3fd554a-a338-42c5-a17d-106465098824 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:32.532593 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f3fd554a-a338-42c5-a17d-106465098824 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:32.533015 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f3fd554a-a338-42c5-a17d-106465098824 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:32.693647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-414d5d87-2bb0-4240-9d4b-2bb0e898f257 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:32.694260 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-414d5d87-2bb0-4240-9d4b-2bb0e898f257 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:32.701169 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-414d5d87-2bb0-4240-9d4b-2bb0e898f257 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:32.701735 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-414d5d87-2bb0-4240-9d4b-2bb0e898f257 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:32.702095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-414d5d87-2bb0-4240-9d4b-2bb0e898f257 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:32.702826 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-414d5d87-2bb0-4240-9d4b-2bb0e898f257 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:32.703280 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-414d5d87-2bb0-4240-9d4b-2bb0e898f257 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:32.703697 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-414d5d87-2bb0-4240-9d4b-2bb0e898f257 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:32.734544 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-f3fd554a-a338-42c5-a17d-106465098824 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:32.743338 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-f3fd554a-a338-42c5-a17d-106465098824 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.223752 Oct 28 09:48:32.743925 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 262/506] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:32 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:32.834449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-414d5d87-2bb0-4240-9d4b-2bb0e898f257 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:32.843174 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-414d5d87-2bb0-4240-9d4b-2bb0e898f257 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:32.844623 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-414d5d87-2bb0-4240-9d4b-2bb0e898f257 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:32.845782 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-414d5d87-2bb0-4240-9d4b-2bb0e898f257 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.156609 Oct 28 09:48:32.846357 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 245/507] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:32 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:33.768026 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b8e79512-f717-48e0-b731-4a07350573e1 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:33.768627 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b8e79512-f717-48e0-b731-4a07350573e1 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:33.775271 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8e79512-f717-48e0-b731-4a07350573e1 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:33.775751 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8e79512-f717-48e0-b731-4a07350573e1 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:33.776182 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8e79512-f717-48e0-b731-4a07350573e1 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:33.776867 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8e79512-f717-48e0-b731-4a07350573e1 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:33.777308 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8e79512-f717-48e0-b731-4a07350573e1 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:33.777722 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8e79512-f717-48e0-b731-4a07350573e1 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:33.870880 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7469c8fa-ad2a-44c6-901f-91d532544492 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:33.871479 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7469c8fa-ad2a-44c6-901f-91d532544492 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:33.878507 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7469c8fa-ad2a-44c6-901f-91d532544492 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:33.878993 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7469c8fa-ad2a-44c6-901f-91d532544492 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:33.879414 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7469c8fa-ad2a-44c6-901f-91d532544492 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:33.880104 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7469c8fa-ad2a-44c6-901f-91d532544492 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:33.880543 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7469c8fa-ad2a-44c6-901f-91d532544492 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:33.880952 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7469c8fa-ad2a-44c6-901f-91d532544492 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:33.938616 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b8e79512-f717-48e0-b731-4a07350573e1 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:33.947221 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b8e79512-f717-48e0-b731-4a07350573e1 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1750 microversion: 2.1 time: 0.182842 Oct 28 09:48:33.947585 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 263/508] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:33 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1750 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:34.031292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7469c8fa-ad2a-44c6-901f-91d532544492 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:34.041674 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7469c8fa-ad2a-44c6-901f-91d532544492 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:34.043415 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7469c8fa-ad2a-44c6-901f-91d532544492 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:34.044635 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7469c8fa-ad2a-44c6-901f-91d532544492 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.177623 Oct 28 09:48:34.045211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 246/509] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:33 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:34.971836 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-752acc2c-90ab-4332-8ae9-67594687dc9e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:34.972438 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-752acc2c-90ab-4332-8ae9-67594687dc9e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:34.979998 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-752acc2c-90ab-4332-8ae9-67594687dc9e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:34.979998 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-752acc2c-90ab-4332-8ae9-67594687dc9e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:34.979998 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-752acc2c-90ab-4332-8ae9-67594687dc9e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:34.980590 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-752acc2c-90ab-4332-8ae9-67594687dc9e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:34.981032 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-752acc2c-90ab-4332-8ae9-67594687dc9e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:34.981444 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-752acc2c-90ab-4332-8ae9-67594687dc9e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:35.072192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f72ad944-7ba8-41d7-a238-66c757fcfe9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:35.072779 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f72ad944-7ba8-41d7-a238-66c757fcfe9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:35.078966 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f72ad944-7ba8-41d7-a238-66c757fcfe9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:35.079444 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f72ad944-7ba8-41d7-a238-66c757fcfe9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:35.079870 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f72ad944-7ba8-41d7-a238-66c757fcfe9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:35.080597 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f72ad944-7ba8-41d7-a238-66c757fcfe9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:35.081035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f72ad944-7ba8-41d7-a238-66c757fcfe9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:35.081451 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f72ad944-7ba8-41d7-a238-66c757fcfe9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:35.138686 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-752acc2c-90ab-4332-8ae9-67594687dc9e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:35.147061 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-752acc2c-90ab-4332-8ae9-67594687dc9e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1755 microversion: 2.1 time: 0.179617 Oct 28 09:48:35.147641 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 264/510] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:34 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1755 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:35.226538 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b487693d-813f-4dac-b442-41ec418e1e06 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:35.226933 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b487693d-813f-4dac-b442-41ec418e1e06 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:35.232866 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b487693d-813f-4dac-b442-41ec418e1e06 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:35.233081 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f72ad944-7ba8-41d7-a238-66c757fcfe9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:35.233342 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b487693d-813f-4dac-b442-41ec418e1e06 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:35.233772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b487693d-813f-4dac-b442-41ec418e1e06 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:35.234495 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b487693d-813f-4dac-b442-41ec418e1e06 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:35.234952 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b487693d-813f-4dac-b442-41ec418e1e06 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:35.235372 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b487693d-813f-4dac-b442-41ec418e1e06 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:35.242594 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f72ad944-7ba8-41d7-a238-66c757fcfe9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:35.244031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f72ad944-7ba8-41d7-a238-66c757fcfe9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:35.245183 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f72ad944-7ba8-41d7-a238-66c757fcfe9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.177476 Oct 28 09:48:35.245738 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 247/511] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:35 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:35.261986 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b487693d-813f-4dac-b442-41ec418e1e06 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Going to try to terminate instance {{(pid=127029) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:48:35.353903 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b487693d-813f-4dac-b442-41ec418e1e06 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 204 len: 0 microversion: 2.1 time: 0.127787 Oct 28 09:48:35.353903 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 265/512] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:48:35 2022] DELETE /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 28 09:48:35.359356 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8d71d61a-3e77-4e0d-9076-36a907db5003 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:35.359967 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8d71d61a-3e77-4e0d-9076-36a907db5003 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:35.366234 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d71d61a-3e77-4e0d-9076-36a907db5003 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:35.366740 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d71d61a-3e77-4e0d-9076-36a907db5003 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:35.367168 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d71d61a-3e77-4e0d-9076-36a907db5003 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:35.367846 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d71d61a-3e77-4e0d-9076-36a907db5003 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:35.368289 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d71d61a-3e77-4e0d-9076-36a907db5003 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:35.368702 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d71d61a-3e77-4e0d-9076-36a907db5003 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:35.498840 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8d71d61a-3e77-4e0d-9076-36a907db5003 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:35.506559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8d71d61a-3e77-4e0d-9076-36a907db5003 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1512 microversion: 2.1 time: 0.150968 Oct 28 09:48:35.507118 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 248/513] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:35 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1512 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:48:36.172026 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b5ef0fd2-6973-4248-8f0f-e793090792e3 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:36.172664 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b5ef0fd2-6973-4248-8f0f-e793090792e3 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:36.179548 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b5ef0fd2-6973-4248-8f0f-e793090792e3 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:36.180019 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b5ef0fd2-6973-4248-8f0f-e793090792e3 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:36.180449 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b5ef0fd2-6973-4248-8f0f-e793090792e3 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:36.181137 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b5ef0fd2-6973-4248-8f0f-e793090792e3 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:36.181573 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b5ef0fd2-6973-4248-8f0f-e793090792e3 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:36.181992 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b5ef0fd2-6973-4248-8f0f-e793090792e3 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:36.270508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d4c7f9e4-9fe2-4462-9ff2-b97b4b0474cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:36.271154 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d4c7f9e4-9fe2-4462-9ff2-b97b4b0474cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:36.278524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d4c7f9e4-9fe2-4462-9ff2-b97b4b0474cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:36.279111 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d4c7f9e4-9fe2-4462-9ff2-b97b4b0474cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:36.280169 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d4c7f9e4-9fe2-4462-9ff2-b97b4b0474cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:36.280866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d4c7f9e4-9fe2-4462-9ff2-b97b4b0474cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:36.281313 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d4c7f9e4-9fe2-4462-9ff2-b97b4b0474cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:36.281732 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d4c7f9e4-9fe2-4462-9ff2-b97b4b0474cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:36.337916 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b5ef0fd2-6973-4248-8f0f-e793090792e3 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:36.346355 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b5ef0fd2-6973-4248-8f0f-e793090792e3 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1755 microversion: 2.1 time: 0.178865 Oct 28 09:48:36.346948 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 266/514] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:36 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1755 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:48:36.422025 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d4c7f9e4-9fe2-4462-9ff2-b97b4b0474cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:36.431115 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d4c7f9e4-9fe2-4462-9ff2-b97b4b0474cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:36.432504 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d4c7f9e4-9fe2-4462-9ff2-b97b4b0474cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:36.433636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d4c7f9e4-9fe2-4462-9ff2-b97b4b0474cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.167228 Oct 28 09:48:36.434180 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 249/515] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:36 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:36.529660 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e99f8398-bfbf-4cdf-b86e-2fe64bc4273f tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:36.530270 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e99f8398-bfbf-4cdf-b86e-2fe64bc4273f tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:36.536714 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e99f8398-bfbf-4cdf-b86e-2fe64bc4273f tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:36.537198 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e99f8398-bfbf-4cdf-b86e-2fe64bc4273f tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:36.537630 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e99f8398-bfbf-4cdf-b86e-2fe64bc4273f tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:36.538319 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e99f8398-bfbf-4cdf-b86e-2fe64bc4273f tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:36.538790 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e99f8398-bfbf-4cdf-b86e-2fe64bc4273f tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:36.539215 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e99f8398-bfbf-4cdf-b86e-2fe64bc4273f tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:36.665285 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e99f8398-bfbf-4cdf-b86e-2fe64bc4273f tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:36.673117 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e99f8398-bfbf-4cdf-b86e-2fe64bc4273f tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1512 microversion: 2.1 time: 0.147351 Oct 28 09:48:36.673734 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 267/516] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:36 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1512 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:37.370050 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-24ae8c10-eee5-48ea-8d9e-c1296706bbf7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:37.370694 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-24ae8c10-eee5-48ea-8d9e-c1296706bbf7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:37.377174 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24ae8c10-eee5-48ea-8d9e-c1296706bbf7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:37.377936 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24ae8c10-eee5-48ea-8d9e-c1296706bbf7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:37.378676 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24ae8c10-eee5-48ea-8d9e-c1296706bbf7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:37.379723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24ae8c10-eee5-48ea-8d9e-c1296706bbf7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:37.380177 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24ae8c10-eee5-48ea-8d9e-c1296706bbf7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:37.380602 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24ae8c10-eee5-48ea-8d9e-c1296706bbf7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:37.459197 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b79a078c-dc6f-4fce-802e-357d510dd05c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:37.459932 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b79a078c-dc6f-4fce-802e-357d510dd05c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:37.466446 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b79a078c-dc6f-4fce-802e-357d510dd05c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:37.466966 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b79a078c-dc6f-4fce-802e-357d510dd05c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:37.467432 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b79a078c-dc6f-4fce-802e-357d510dd05c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:37.468136 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b79a078c-dc6f-4fce-802e-357d510dd05c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:37.468587 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b79a078c-dc6f-4fce-802e-357d510dd05c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:37.469009 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b79a078c-dc6f-4fce-802e-357d510dd05c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:37.560765 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-24ae8c10-eee5-48ea-8d9e-c1296706bbf7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:37.571754 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-24ae8c10-eee5-48ea-8d9e-c1296706bbf7 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1755 microversion: 2.1 time: 0.204987 Oct 28 09:48:37.572481 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 250/517] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:37 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1755 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:37.620373 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b79a078c-dc6f-4fce-802e-357d510dd05c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:37.628833 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b79a078c-dc6f-4fce-802e-357d510dd05c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:37.630272 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b79a078c-dc6f-4fce-802e-357d510dd05c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:37.631457 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b79a078c-dc6f-4fce-802e-357d510dd05c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.176296 Oct 28 09:48:37.632081 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 268/518] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:37 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:37.696220 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7fcfa102-044d-4e35-8b06-f77f8ee907a4 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:37.696816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7fcfa102-044d-4e35-8b06-f77f8ee907a4 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:37.702748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7fcfa102-044d-4e35-8b06-f77f8ee907a4 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:37.703223 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7fcfa102-044d-4e35-8b06-f77f8ee907a4 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:37.703645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7fcfa102-044d-4e35-8b06-f77f8ee907a4 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:37.704318 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7fcfa102-044d-4e35-8b06-f77f8ee907a4 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:37.704762 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7fcfa102-044d-4e35-8b06-f77f8ee907a4 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:37.705174 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7fcfa102-044d-4e35-8b06-f77f8ee907a4 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:37.879617 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7fcfa102-044d-4e35-8b06-f77f8ee907a4 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:37.887357 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7fcfa102-044d-4e35-8b06-f77f8ee907a4 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1512 microversion: 2.1 time: 0.194951 Oct 28 09:48:37.887924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 251/519] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:37 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1512 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:38.345087 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-91366000-07ce-4d0f-b1e3-e4e4cb70fc80 req-9349130d-2cf7-4d05-b5e6-93b454930ff9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a50a18b7-b11e-48cf-b8dc-58d39afc4d27", "name": "network-vif-deleted", "tag": "1c436b7e-fe82-419c-9976-f87b1fa4439c"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:48:38.359582 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-91366000-07ce-4d0f-b1e3-e4e4cb70fc80 req-9349130d-2cf7-4d05-b5e6-93b454930ff9 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:38.360276 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-91366000-07ce-4d0f-b1e3-e4e4cb70fc80 req-9349130d-2cf7-4d05-b5e6-93b454930ff9 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:38.360671 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-91366000-07ce-4d0f-b1e3-e4e4cb70fc80 req-9349130d-2cf7-4d05-b5e6-93b454930ff9 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:38.372379 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-91366000-07ce-4d0f-b1e3-e4e4cb70fc80 req-9349130d-2cf7-4d05-b5e6-93b454930ff9 service nova] Creating event network-vif-deleted:1c436b7e-fe82-419c-9976-f87b1fa4439c for instance a50a18b7-b11e-48cf-b8dc-58d39afc4d27 on hv-cinder-zed-3 Oct 28 09:48:38.377671 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-91366000-07ce-4d0f-b1e3-e4e4cb70fc80 req-9349130d-2cf7-4d05-b5e6-93b454930ff9 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036168 Oct 28 09:48:38.378604 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 269/520] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:48:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:48:38.597107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-cefc9d4d-d196-4669-9955-0320cf61bf0b tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:38.598032 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-cefc9d4d-d196-4669-9955-0320cf61bf0b tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:38.604384 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cefc9d4d-d196-4669-9955-0320cf61bf0b tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:38.604855 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cefc9d4d-d196-4669-9955-0320cf61bf0b tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:38.605279 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cefc9d4d-d196-4669-9955-0320cf61bf0b tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:38.605979 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cefc9d4d-d196-4669-9955-0320cf61bf0b tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:38.606612 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cefc9d4d-d196-4669-9955-0320cf61bf0b tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:38.606844 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cefc9d4d-d196-4669-9955-0320cf61bf0b tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:38.611010 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:38.657596 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d1289f83-c7cb-479c-912e-5b2057e1036c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:38.658274 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d1289f83-c7cb-479c-912e-5b2057e1036c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:38.664953 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d1289f83-c7cb-479c-912e-5b2057e1036c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:38.665426 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d1289f83-c7cb-479c-912e-5b2057e1036c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:38.665851 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d1289f83-c7cb-479c-912e-5b2057e1036c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:38.666568 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d1289f83-c7cb-479c-912e-5b2057e1036c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:38.667016 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d1289f83-c7cb-479c-912e-5b2057e1036c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:38.667429 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d1289f83-c7cb-479c-912e-5b2057e1036c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:38.807556 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-cefc9d4d-d196-4669-9955-0320cf61bf0b tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:38.816397 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-cefc9d4d-d196-4669-9955-0320cf61bf0b tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1755 microversion: 2.1 time: 0.224056 Oct 28 09:48:38.816966 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 252/521] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:38 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1755 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:38.836297 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d1289f83-c7cb-479c-912e-5b2057e1036c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:38.844911 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d1289f83-c7cb-479c-912e-5b2057e1036c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:38.846355 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d1289f83-c7cb-479c-912e-5b2057e1036c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:38.847525 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d1289f83-c7cb-479c-912e-5b2057e1036c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.194665 Oct 28 09:48:38.848077 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 270/522] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:38 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:38.910228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:38.910846 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:38.916757 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:38.917230 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:38.917653 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:38.918616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:38.919058 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:38.919467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:38.949729 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:38.950194 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:38.950660 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:38.951101 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lazy-loading 'fault' on Instance uuid a50a18b7-b11e-48cf-b8dc-58d39afc4d27 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:48:38.993346 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:39.000909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-138e4e38-f89b-40db-8b4c-52c5ad62143d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1307 microversion: 2.1 time: 0.094512 Oct 28 09:48:39.001467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 253/523] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:38 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1307 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:39.840202 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8a524ebf-c232-4451-b044-0090381c6930 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:39.840810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8a524ebf-c232-4451-b044-0090381c6930 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:39.848084 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8a524ebf-c232-4451-b044-0090381c6930 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:39.848563 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8a524ebf-c232-4451-b044-0090381c6930 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:39.849030 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8a524ebf-c232-4451-b044-0090381c6930 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:39.849678 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8a524ebf-c232-4451-b044-0090381c6930 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:39.850154 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8a524ebf-c232-4451-b044-0090381c6930 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:39.850603 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8a524ebf-c232-4451-b044-0090381c6930 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:39.872408 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2c8f231e-e075-4bfc-b77d-4ae3d22bbe20 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:39.873767 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2c8f231e-e075-4bfc-b77d-4ae3d22bbe20 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:39.883188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2c8f231e-e075-4bfc-b77d-4ae3d22bbe20 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:39.884059 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2c8f231e-e075-4bfc-b77d-4ae3d22bbe20 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:39.884358 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2c8f231e-e075-4bfc-b77d-4ae3d22bbe20 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:39.885144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2c8f231e-e075-4bfc-b77d-4ae3d22bbe20 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:39.885598 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2c8f231e-e075-4bfc-b77d-4ae3d22bbe20 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:39.886178 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2c8f231e-e075-4bfc-b77d-4ae3d22bbe20 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:40.057739 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-8a524ebf-c232-4451-b044-0090381c6930 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:40.065786 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8a524ebf-c232-4451-b044-0090381c6930 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1755 microversion: 2.1 time: 0.229361 Oct 28 09:48:40.066632 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 271/524] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:39 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1755 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:40.070895 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:40.071503 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:40.073889 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2c8f231e-e075-4bfc-b77d-4ae3d22bbe20 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:40.077664 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:40.078343 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:40.078802 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:40.079538 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:40.079944 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:40.080893 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:40.083480 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2c8f231e-e075-4bfc-b77d-4ae3d22bbe20 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:40.084950 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2c8f231e-e075-4bfc-b77d-4ae3d22bbe20 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:40.086044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2c8f231e-e075-4bfc-b77d-4ae3d22bbe20 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.217440 Oct 28 09:48:40.086618 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 254/525] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:39 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:40.099627 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:40.115819 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:40.116318 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:40.116890 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:40.117345 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.objects.instance [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lazy-loading 'fault' on Instance uuid a50a18b7-b11e-48cf-b8dc-58d39afc4d27 {{(pid=127029) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:48:40.164189 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:40.172146 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2fa6b459-8d06-438d-a381-d887e5e334e6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 200 len: 1307 microversion: 2.1 time: 0.104842 Oct 28 09:48:40.172711 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 272/526] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:40 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 1307 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:41.089855 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-047e41b6-1c22-453d-8a64-ead90b2b8236 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:41.090488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-047e41b6-1c22-453d-8a64-ead90b2b8236 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:41.097398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-047e41b6-1c22-453d-8a64-ead90b2b8236 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:41.097875 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-047e41b6-1c22-453d-8a64-ead90b2b8236 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:41.098297 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-047e41b6-1c22-453d-8a64-ead90b2b8236 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:41.099006 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-047e41b6-1c22-453d-8a64-ead90b2b8236 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:41.099451 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-047e41b6-1c22-453d-8a64-ead90b2b8236 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:41.099875 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-047e41b6-1c22-453d-8a64-ead90b2b8236 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:41.111720 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-45dfab04-be74-442d-86e5-641e8b687959 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:41.112873 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-45dfab04-be74-442d-86e5-641e8b687959 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:41.119464 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-45dfab04-be74-442d-86e5-641e8b687959 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:41.119943 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-45dfab04-be74-442d-86e5-641e8b687959 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:41.120370 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-45dfab04-be74-442d-86e5-641e8b687959 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:41.121047 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-45dfab04-be74-442d-86e5-641e8b687959 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:41.121488 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-45dfab04-be74-442d-86e5-641e8b687959 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:41.121913 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-45dfab04-be74-442d-86e5-641e8b687959 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:41.313763 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-45dfab04-be74-442d-86e5-641e8b687959 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:41.318753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-047e41b6-1c22-453d-8a64-ead90b2b8236 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:41.327417 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-047e41b6-1c22-453d-8a64-ead90b2b8236 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1755 microversion: 2.1 time: 0.241291 Oct 28 09:48:41.328352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 255/527] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:41 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1755 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:41.333653 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-898f7380-d373-48c1-a10c-e66bae6ccd81 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:41.334363 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-898f7380-d373-48c1-a10c-e66bae6ccd81 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: a50a18b7-b11e-48cf-b8dc-58d39afc4d27] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:41.340243 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-898f7380-d373-48c1-a10c-e66bae6ccd81 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:41.340665 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-898f7380-d373-48c1-a10c-e66bae6ccd81 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:41.341098 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-898f7380-d373-48c1-a10c-e66bae6ccd81 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:41.341791 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-898f7380-d373-48c1-a10c-e66bae6ccd81 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:41.342238 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-898f7380-d373-48c1-a10c-e66bae6ccd81 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:41.342689 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-898f7380-d373-48c1-a10c-e66bae6ccd81 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:41.343362 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-45dfab04-be74-442d-86e5-641e8b687959 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:41.344949 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-45dfab04-be74-442d-86e5-641e8b687959 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:41.348044 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-45dfab04-be74-442d-86e5-641e8b687959 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.238284 Oct 28 09:48:41.348727 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 273/528] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:41 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:41.357137 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-898f7380-d373-48c1-a10c-e66bae6ccd81 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] HTTP exception thrown: Instance a50a18b7-b11e-48cf-b8dc-58d39afc4d27 could not be found. Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-898f7380-d373-48c1-a10c-e66bae6ccd81 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Request method failure captured: Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 HTTP/1.1 Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6UnJ0XS3DcbOAGlBvcokWOetHelVSNGAy0AA6ILn39s4_F70yEsLs6CjMI8eDJZzaXK_Vh-4HzBHeo6e1LWtOD2BwJHIifsVVqLySG6fHdXxgyxwCYeB2dy8v6CsCqwEus9FQmwu6YybRfmrg_mGOBhBtGsf1kxUvoqPIgxqYbOq4w Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 9a20b51b10ce463da8f9ad280063bb09 Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1142097989 Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:48:41.358807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:48:41.360365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/9a20b51b10ce463da8f9ad280063bb09"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/9a20b51b10ce463da8f9ad280063bb09"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_9a20b51b10ce463da8f9ad280063bb09"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/9a20b51b10ce463da8f9ad280063bb09"}]}] Oct 28 09:48:41.360365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-VolumesSnapshotTestJSON-1142097989 Oct 28 09:48:41.360365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 9a20b51b10ce463da8f9ad280063bb09 Oct 28 09:48:41.360365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1142097989 Oct 28 09:48:41.360365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-VolumesSnapshotTestJSON-1142097989-project Oct 28 09:48:41.360365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:48:41.360365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:48:41.360365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: ec23c6593e7a446298d607294997664c Oct 28 09:48:41.360365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-VolumesSnapshotTestJSON-1142097989-project Oct 28 09:48:41.360365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:48:41.360365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance a50a18b7-b11e-48cf-b8dc-58d39afc4d27 could not be found. Oct 28 09:48:41.360365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'a50a18b7-b11e-48cf-b8dc-58d39afc4d27'} Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a50a18b7-b11e-48cf-b8dc-58d39afc4d27 could not be found. Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:48:41.361188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a50a18b7-b11e-48cf-b8dc-58d39afc4d27 could not be found. Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a50a18b7-b11e-48cf-b8dc-58d39afc4d27 could not be found. Oct 28 09:48:41.362476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:48:41.363760 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-898f7380-d373-48c1-a10c-e66bae6ccd81 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Returning 404 to user: Instance a50a18b7-b11e-48cf-b8dc-58d39afc4d27 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:48:41.363760 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-898f7380-d373-48c1-a10c-e66bae6ccd81 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27" status: 404 len: 111 microversion: 2.1 time: 0.030974 Oct 28 09:48:41.363760 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 256/529] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:41 2022] GET /compute/v2.1/servers/a50a18b7-b11e-48cf-b8dc-58d39afc4d27 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:48:42.356397 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-4904992e-3641-41ec-8a8d-26ff8872057c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:42.357011 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-4904992e-3641-41ec-8a8d-26ff8872057c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:42.363978 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4904992e-3641-41ec-8a8d-26ff8872057c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:42.364453 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4904992e-3641-41ec-8a8d-26ff8872057c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:42.364881 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4904992e-3641-41ec-8a8d-26ff8872057c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:42.365557 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4904992e-3641-41ec-8a8d-26ff8872057c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:42.365994 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4904992e-3641-41ec-8a8d-26ff8872057c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:42.366434 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4904992e-3641-41ec-8a8d-26ff8872057c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:42.373018 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a5da769a-30fd-4727-805e-198fd70eff10 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:42.373625 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a5da769a-30fd-4727-805e-198fd70eff10 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:42.381290 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a5da769a-30fd-4727-805e-198fd70eff10 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:42.382479 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a5da769a-30fd-4727-805e-198fd70eff10 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:42.382921 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a5da769a-30fd-4727-805e-198fd70eff10 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:42.383623 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a5da769a-30fd-4727-805e-198fd70eff10 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:42.384071 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a5da769a-30fd-4727-805e-198fd70eff10 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:42.384490 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a5da769a-30fd-4727-805e-198fd70eff10 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:42.574882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a5da769a-30fd-4727-805e-198fd70eff10 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:42.579797 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-4904992e-3641-41ec-8a8d-26ff8872057c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:42.585801 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a5da769a-30fd-4727-805e-198fd70eff10 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:42.587332 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a5da769a-30fd-4727-805e-198fd70eff10 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:42.588328 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-4904992e-3641-41ec-8a8d-26ff8872057c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1755 microversion: 2.1 time: 0.235840 Oct 28 09:48:42.588471 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a5da769a-30fd-4727-805e-198fd70eff10 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.219292 Oct 28 09:48:42.588860 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 274/530] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:42 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1755 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:42.589231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 257/531] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:42 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:43.612385 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-f90b22f6-2436-4a8b-82b1-4e4f378d28b9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:43.613144 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-f90b22f6-2436-4a8b-82b1-4e4f378d28b9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:43.614933 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d70307b5-ffe3-4e8f-a74b-4d5e6409aacd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:43.615497 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=63,nova_api:UPDATE=1 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:43.615617 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d70307b5-ffe3-4e8f-a74b-4d5e6409aacd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:43.621058 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f90b22f6-2436-4a8b-82b1-4e4f378d28b9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:43.621535 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f90b22f6-2436-4a8b-82b1-4e4f378d28b9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:43.621813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d70307b5-ffe3-4e8f-a74b-4d5e6409aacd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:43.621962 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f90b22f6-2436-4a8b-82b1-4e4f378d28b9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:43.622285 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d70307b5-ffe3-4e8f-a74b-4d5e6409aacd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:43.622669 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f90b22f6-2436-4a8b-82b1-4e4f378d28b9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:43.622771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d70307b5-ffe3-4e8f-a74b-4d5e6409aacd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:43.623158 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f90b22f6-2436-4a8b-82b1-4e4f378d28b9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:43.623565 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d70307b5-ffe3-4e8f-a74b-4d5e6409aacd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:43.623671 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f90b22f6-2436-4a8b-82b1-4e4f378d28b9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:43.624125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d70307b5-ffe3-4e8f-a74b-4d5e6409aacd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:43.624430 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d70307b5-ffe3-4e8f-a74b-4d5e6409aacd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:43.829258 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d70307b5-ffe3-4e8f-a74b-4d5e6409aacd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:43.834312 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-f90b22f6-2436-4a8b-82b1-4e4f378d28b9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:43.837893 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d70307b5-ffe3-4e8f-a74b-4d5e6409aacd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:43.839377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d70307b5-ffe3-4e8f-a74b-4d5e6409aacd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:43.840526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d70307b5-ffe3-4e8f-a74b-4d5e6409aacd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.230192 Oct 28 09:48:43.841073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 258/532] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:43 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:43.842333 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-f90b22f6-2436-4a8b-82b1-4e4f378d28b9 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1755 microversion: 2.1 time: 0.234526 Oct 28 09:48:43.842909 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 275/533] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:43 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1755 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:44.865777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3162b19e-7656-4d02-8f22-3fba32b10047 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:44.866370 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e19244ff-8a4b-48b6-938d-0f7e17e7536d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:44.866759 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3162b19e-7656-4d02-8f22-3fba32b10047 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:44.866848 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e19244ff-8a4b-48b6-938d-0f7e17e7536d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:44.873387 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e19244ff-8a4b-48b6-938d-0f7e17e7536d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:44.873588 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3162b19e-7656-4d02-8f22-3fba32b10047 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:44.873868 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e19244ff-8a4b-48b6-938d-0f7e17e7536d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:44.874039 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3162b19e-7656-4d02-8f22-3fba32b10047 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:44.874291 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e19244ff-8a4b-48b6-938d-0f7e17e7536d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:44.874487 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3162b19e-7656-4d02-8f22-3fba32b10047 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:44.875009 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e19244ff-8a4b-48b6-938d-0f7e17e7536d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:44.875183 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3162b19e-7656-4d02-8f22-3fba32b10047 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:44.875456 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e19244ff-8a4b-48b6-938d-0f7e17e7536d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:44.875628 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3162b19e-7656-4d02-8f22-3fba32b10047 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:44.875872 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e19244ff-8a4b-48b6-938d-0f7e17e7536d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:44.876051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3162b19e-7656-4d02-8f22-3fba32b10047 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:45.082184 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e19244ff-8a4b-48b6-938d-0f7e17e7536d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:45.091645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3162b19e-7656-4d02-8f22-3fba32b10047 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:45.100176 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3162b19e-7656-4d02-8f22-3fba32b10047 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1755 microversion: 2.1 time: 0.238667 Oct 28 09:48:45.106283 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e19244ff-8a4b-48b6-938d-0f7e17e7536d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:45.107888 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e19244ff-8a4b-48b6-938d-0f7e17e7536d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:45.109060 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e19244ff-8a4b-48b6-938d-0f7e17e7536d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.247323 Oct 28 09:48:45.109615 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 276/534] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:44 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:45.114615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 259/535] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:44 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1755 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:45.115025 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-013cb68c-9741-4d30-8637-e7f62fe30d82 req-6ab19819-3118-43b2-a1a4-6fbdd7e0a25d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c716d5fd-f257-4f6a-8f52-18ad59e5c31e", "name": "network-vif-plugged", "status": "completed", "tag": "c2310b44-82bb-408f-85b5-8a5556b92e25"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:48:45.124394 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-013cb68c-9741-4d30-8637-e7f62fe30d82 req-6ab19819-3118-43b2-a1a4-6fbdd7e0a25d service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:45.124803 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-013cb68c-9741-4d30-8637-e7f62fe30d82 req-6ab19819-3118-43b2-a1a4-6fbdd7e0a25d service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:45.125228 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-013cb68c-9741-4d30-8637-e7f62fe30d82 req-6ab19819-3118-43b2-a1a4-6fbdd7e0a25d service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:45.136140 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-013cb68c-9741-4d30-8637-e7f62fe30d82 req-6ab19819-3118-43b2-a1a4-6fbdd7e0a25d service nova] Creating event network-vif-plugged:c2310b44-82bb-408f-85b5-8a5556b92e25 for instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e on hv-cinder-zed-3 Oct 28 09:48:45.141135 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-013cb68c-9741-4d30-8637-e7f62fe30d82 req-6ab19819-3118-43b2-a1a4-6fbdd7e0a25d service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030100 Oct 28 09:48:45.141938 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 277/536] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:48:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:48:46.123651 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d57c1ce6-4fa0-45f8-a85b-83f59fa4e95a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:46.124189 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d57c1ce6-4fa0-45f8-a85b-83f59fa4e95a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:46.130546 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d57c1ce6-4fa0-45f8-a85b-83f59fa4e95a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:46.131016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d57c1ce6-4fa0-45f8-a85b-83f59fa4e95a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:46.131440 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d57c1ce6-4fa0-45f8-a85b-83f59fa4e95a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:46.132114 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d57c1ce6-4fa0-45f8-a85b-83f59fa4e95a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:46.132556 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d57c1ce6-4fa0-45f8-a85b-83f59fa4e95a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:46.132968 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d57c1ce6-4fa0-45f8-a85b-83f59fa4e95a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:46.137015 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-33f67289-5715-46ee-9dce-56ed6a633927 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:46.137710 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-33f67289-5715-46ee-9dce-56ed6a633927 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:46.144244 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-33f67289-5715-46ee-9dce-56ed6a633927 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:46.144723 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-33f67289-5715-46ee-9dce-56ed6a633927 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:46.145156 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-33f67289-5715-46ee-9dce-56ed6a633927 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:46.145910 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-33f67289-5715-46ee-9dce-56ed6a633927 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:46.146499 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-33f67289-5715-46ee-9dce-56ed6a633927 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:46.146849 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-33f67289-5715-46ee-9dce-56ed6a633927 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:46.552195 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-33f67289-5715-46ee-9dce-56ed6a633927 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:46.557137 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d57c1ce6-4fa0-45f8-a85b-83f59fa4e95a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:46.561653 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-33f67289-5715-46ee-9dce-56ed6a633927 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:46.563137 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-33f67289-5715-46ee-9dce-56ed6a633927 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:46.564329 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-33f67289-5715-46ee-9dce-56ed6a633927 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.432121 Oct 28 09:48:46.565477 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 278/537] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:46 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:46.566079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d57c1ce6-4fa0-45f8-a85b-83f59fa4e95a tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1755 microversion: 2.1 time: 0.446306 Oct 28 09:48:46.566858 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 260/538] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:46 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1755 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:47.590708 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-9bbcb67a-3ba1-42f0-9af5-13c4617569a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:47.591288 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-cfe8908f-495c-4f25-aab6-b1bd8c70084a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:47.591649 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-9bbcb67a-3ba1-42f0-9af5-13c4617569a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:47.591745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-cfe8908f-495c-4f25-aab6-b1bd8c70084a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:47.593829 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=62 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:47.598574 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9bbcb67a-3ba1-42f0-9af5-13c4617569a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:47.599069 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9bbcb67a-3ba1-42f0-9af5-13c4617569a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:47.599505 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9bbcb67a-3ba1-42f0-9af5-13c4617569a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:47.600141 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cfe8908f-495c-4f25-aab6-b1bd8c70084a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:47.600254 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9bbcb67a-3ba1-42f0-9af5-13c4617569a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:47.600629 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cfe8908f-495c-4f25-aab6-b1bd8c70084a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:47.600753 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9bbcb67a-3ba1-42f0-9af5-13c4617569a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:47.602064 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9bbcb67a-3ba1-42f0-9af5-13c4617569a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:47.602181 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cfe8908f-495c-4f25-aab6-b1bd8c70084a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:47.602181 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cfe8908f-495c-4f25-aab6-b1bd8c70084a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:47.602181 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cfe8908f-495c-4f25-aab6-b1bd8c70084a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:47.602596 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cfe8908f-495c-4f25-aab6-b1bd8c70084a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:47.803625 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-cfe8908f-495c-4f25-aab6-b1bd8c70084a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:47.808771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-9bbcb67a-3ba1-42f0-9af5-13c4617569a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:47.812243 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-cfe8908f-495c-4f25-aab6-b1bd8c70084a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:47.813683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-cfe8908f-495c-4f25-aab6-b1bd8c70084a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:47.815390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-cfe8908f-495c-4f25-aab6-b1bd8c70084a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.228135 Oct 28 09:48:47.815946 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 261/539] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:47 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:48:47.817583 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-9bbcb67a-3ba1-42f0-9af5-13c4617569a5 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1755 microversion: 2.1 time: 0.230788 Oct 28 09:48:47.818149 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 279/540] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:47 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1755 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:48.840967 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-14f9564e-dd1f-45f6-a2d1-bdba4bcd32a4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:48.841587 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3ce52b4b-60ea-422f-998a-8eb9d75fcb4f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:48.841947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-14f9564e-dd1f-45f6-a2d1-bdba4bcd32a4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:48.842039 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-3ce52b4b-60ea-422f-998a-8eb9d75fcb4f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:48.848083 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3ce52b4b-60ea-422f-998a-8eb9d75fcb4f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:48.848246 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-14f9564e-dd1f-45f6-a2d1-bdba4bcd32a4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:48.848561 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3ce52b4b-60ea-422f-998a-8eb9d75fcb4f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:48.848672 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-14f9564e-dd1f-45f6-a2d1-bdba4bcd32a4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:48.848994 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3ce52b4b-60ea-422f-998a-8eb9d75fcb4f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:48.849114 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-14f9564e-dd1f-45f6-a2d1-bdba4bcd32a4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:48.849680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-14f9564e-dd1f-45f6-a2d1-bdba4bcd32a4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:48.849788 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3ce52b4b-60ea-422f-998a-8eb9d75fcb4f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:48.850129 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-14f9564e-dd1f-45f6-a2d1-bdba4bcd32a4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:48.850223 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3ce52b4b-60ea-422f-998a-8eb9d75fcb4f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:48.850575 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-14f9564e-dd1f-45f6-a2d1-bdba4bcd32a4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:48.850681 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3ce52b4b-60ea-422f-998a-8eb9d75fcb4f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:49.055927 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3ce52b4b-60ea-422f-998a-8eb9d75fcb4f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:49.060819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-14f9564e-dd1f-45f6-a2d1-bdba4bcd32a4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:49.064620 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3ce52b4b-60ea-422f-998a-8eb9d75fcb4f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:49.066666 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3ce52b4b-60ea-422f-998a-8eb9d75fcb4f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:49.067812 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-3ce52b4b-60ea-422f-998a-8eb9d75fcb4f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.230722 Oct 28 09:48:49.068366 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 280/541] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:48 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:49.070041 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-14f9564e-dd1f-45f6-a2d1-bdba4bcd32a4 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1755 microversion: 2.1 time: 0.232897 Oct 28 09:48:49.070631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 262/542] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:48 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1755 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:50.095560 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-678294c4-ff3b-4b00-a69f-9d5ba84af0c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:50.096607 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9daf17ae-8ff7-4cc6-89a7-42dd6a1c2c79 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:50.097054 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-678294c4-ff3b-4b00-a69f-9d5ba84af0c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:50.097660 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9daf17ae-8ff7-4cc6-89a7-42dd6a1c2c79 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:50.107425 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-678294c4-ff3b-4b00-a69f-9d5ba84af0c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:50.107923 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9daf17ae-8ff7-4cc6-89a7-42dd6a1c2c79 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:50.108195 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-678294c4-ff3b-4b00-a69f-9d5ba84af0c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:50.108694 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9daf17ae-8ff7-4cc6-89a7-42dd6a1c2c79 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:50.108899 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-678294c4-ff3b-4b00-a69f-9d5ba84af0c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:50.109373 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9daf17ae-8ff7-4cc6-89a7-42dd6a1c2c79 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:50.110019 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-678294c4-ff3b-4b00-a69f-9d5ba84af0c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:50.110675 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9daf17ae-8ff7-4cc6-89a7-42dd6a1c2c79 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:50.110798 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-678294c4-ff3b-4b00-a69f-9d5ba84af0c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:50.111413 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9daf17ae-8ff7-4cc6-89a7-42dd6a1c2c79 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:50.111537 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-678294c4-ff3b-4b00-a69f-9d5ba84af0c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:50.112663 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9daf17ae-8ff7-4cc6-89a7-42dd6a1c2c79 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:50.315573 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-678294c4-ff3b-4b00-a69f-9d5ba84af0c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:50.324275 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-678294c4-ff3b-4b00-a69f-9d5ba84af0c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:50.325623 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-9daf17ae-8ff7-4cc6-89a7-42dd6a1c2c79 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:50.325755 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-678294c4-ff3b-4b00-a69f-9d5ba84af0c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:50.326873 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-678294c4-ff3b-4b00-a69f-9d5ba84af0c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1808 microversion: 2.20 time: 0.237971 Oct 28 09:48:50.327418 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 281/543] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:50 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1808 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:50.334697 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9daf17ae-8ff7-4cc6-89a7-42dd6a1c2c79 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1755 microversion: 2.1 time: 0.245229 Oct 28 09:48:50.335265 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 263/544] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:50 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1755 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:51.338689 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1384674417", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "networks": [{"uuid": "e6e285c9-7f9a-43e5-8517-efe4e78507ee"}]}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:48:51.356388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6d7e87e8-c3db-492c-8e45-6cf214be13fc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:51.356986 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6d7e87e8-c3db-492c-8e45-6cf214be13fc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:51.363668 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d7e87e8-c3db-492c-8e45-6cf214be13fc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:51.364147 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d7e87e8-c3db-492c-8e45-6cf214be13fc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:51.364573 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d7e87e8-c3db-492c-8e45-6cf214be13fc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:51.365256 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d7e87e8-c3db-492c-8e45-6cf214be13fc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:51.365722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d7e87e8-c3db-492c-8e45-6cf214be13fc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:51.366133 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d7e87e8-c3db-492c-8e45-6cf214be13fc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:51.395354 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.network.neutron [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] validate_networks() for [('e6e285c9-7f9a-43e5-8517-efe4e78507ee', None, None, None, None, None)] {{(pid=127029) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:48:51.589624 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6d7e87e8-c3db-492c-8e45-6cf214be13fc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:51.598737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6d7e87e8-c3db-492c-8e45-6cf214be13fc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:51.600177 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6d7e87e8-c3db-492c-8e45-6cf214be13fc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:51.601342 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6d7e87e8-c3db-492c-8e45-6cf214be13fc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1795 microversion: 2.20 time: 0.251051 Oct 28 09:48:51.601900 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 264/545] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:48:51 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1795 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:48:51.606695 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-bf3c6f08-b5c5-47d3-898f-9a968ec9c182 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:51.607240 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-bf3c6f08-b5c5-47d3-898f-9a968ec9c182 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:51.613161 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bf3c6f08-b5c5-47d3-898f-9a968ec9c182 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:51.613649 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bf3c6f08-b5c5-47d3-898f-9a968ec9c182 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:51.614080 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bf3c6f08-b5c5-47d3-898f-9a968ec9c182 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:51.614801 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bf3c6f08-b5c5-47d3-898f-9a968ec9c182 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:51.615247 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bf3c6f08-b5c5-47d3-898f-9a968ec9c182 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:51.615664 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bf3c6f08-b5c5-47d3-898f-9a968ec9c182 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:51.869647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-bf3c6f08-b5c5-47d3-898f-9a968ec9c182 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:51.878771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-bf3c6f08-b5c5-47d3-898f-9a968ec9c182 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1758 microversion: 2.1 time: 0.275669 Oct 28 09:48:51.879346 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 265/546] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:51 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1758 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:51.905643 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Flavor limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:48:51.906057 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Image limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:48:51.906551 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Flavor pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:48:51.906952 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Image pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:48:51.907352 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:48:51.917812 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:51.918286 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:51.919035 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:51.919755 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:51.920189 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:51.920603 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:51.939641 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Getting quotas for project 9a20b51b10ce463da8f9ad280063bb09. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:48:51.944227 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Getting quotas for user ec23c6593e7a446298d607294997664c and project 9a20b51b10ce463da8f9ad280063bb09. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:48:51.955608 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Going to run 1 instances... {{(pid=127029) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:48:51.957895 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127029) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:48:52.006375 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-983c9bd7-61e3-4779-8dd8-e4dfb68c2b5a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.671729 Oct 28 09:48:52.007032 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 282/547] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:48:51 2022] POST /compute/v2.1/servers => generated 380 bytes in 673 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 09:48:52.022085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-533f191b-5031-4e67-8ced-672a0ae4a14d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:52.023159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-533f191b-5031-4e67-8ced-672a0ae4a14d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:52.081774 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-533f191b-5031-4e67-8ced-672a0ae4a14d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:52.088692 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-533f191b-5031-4e67-8ced-672a0ae4a14d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1219 microversion: 2.1 time: 0.073925 Oct 28 09:48:52.089447 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 266/548] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:52 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1219 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:53.112327 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-1b7d5eea-479c-47f2-8c7b-b879b30846ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:53.112932 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-1b7d5eea-479c-47f2-8c7b-b879b30846ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:53.122260 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1b7d5eea-479c-47f2-8c7b-b879b30846ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:53.122803 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1b7d5eea-479c-47f2-8c7b-b879b30846ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:53.123270 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1b7d5eea-479c-47f2-8c7b-b879b30846ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:53.123963 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1b7d5eea-479c-47f2-8c7b-b879b30846ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:53.124410 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1b7d5eea-479c-47f2-8c7b-b879b30846ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:53.124833 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1b7d5eea-479c-47f2-8c7b-b879b30846ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:53.185948 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-1b7d5eea-479c-47f2-8c7b-b879b30846ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:53.194328 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-1b7d5eea-479c-47f2-8c7b-b879b30846ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1271 microversion: 2.1 time: 0.086488 Oct 28 09:48:53.194939 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 283/549] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:53 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1271 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:54.219449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-01a9d68d-a9b5-4424-8d72-d7c97c42838c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:54.220167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-01a9d68d-a9b5-4424-8d72-d7c97c42838c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:54.227698 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-01a9d68d-a9b5-4424-8d72-d7c97c42838c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:54.228177 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-01a9d68d-a9b5-4424-8d72-d7c97c42838c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:54.228614 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-01a9d68d-a9b5-4424-8d72-d7c97c42838c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:54.229306 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-01a9d68d-a9b5-4424-8d72-d7c97c42838c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:54.229753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-01a9d68d-a9b5-4424-8d72-d7c97c42838c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:54.230170 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-01a9d68d-a9b5-4424-8d72-d7c97c42838c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:54.437204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-01a9d68d-a9b5-4424-8d72-d7c97c42838c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:54.445438 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-01a9d68d-a9b5-4424-8d72-d7c97c42838c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1319 microversion: 2.1 time: 0.229816 Oct 28 09:48:54.446009 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 267/550] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:54 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1319 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:54.822879 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-3828f5bc-66b3-4fc8-9387-67b43c3bb440 req-f5b2ca54-c13c-4733-ab5f-f1a412b88630 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "85327eca-6324-41a4-8b03-f2d5226cf2f5", "tag": "be115348-3000-4d12-9056-f68416196802"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:48:54.839173 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-3828f5bc-66b3-4fc8-9387-67b43c3bb440 req-f5b2ca54-c13c-4733-ab5f-f1a412b88630 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:54.839823 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-3828f5bc-66b3-4fc8-9387-67b43c3bb440 req-f5b2ca54-c13c-4733-ab5f-f1a412b88630 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:54.840256 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-3828f5bc-66b3-4fc8-9387-67b43c3bb440 req-f5b2ca54-c13c-4733-ab5f-f1a412b88630 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:54.853007 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-3828f5bc-66b3-4fc8-9387-67b43c3bb440 req-f5b2ca54-c13c-4733-ab5f-f1a412b88630 service nova] Creating event network-changed:be115348-3000-4d12-9056-f68416196802 for instance 85327eca-6324-41a4-8b03-f2d5226cf2f5 on hv-cinder-zed-3 Oct 28 09:48:54.857851 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-3828f5bc-66b3-4fc8-9387-67b43c3bb440 req-f5b2ca54-c13c-4733-ab5f-f1a412b88630 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038699 Oct 28 09:48:54.858514 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 284/551] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:48:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:48:55.468877 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f8156c46-709f-4aa8-b010-52e8d85b69a2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:55.469393 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f8156c46-709f-4aa8-b010-52e8d85b69a2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:55.475792 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f8156c46-709f-4aa8-b010-52e8d85b69a2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:55.476267 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f8156c46-709f-4aa8-b010-52e8d85b69a2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:55.476699 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f8156c46-709f-4aa8-b010-52e8d85b69a2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:55.477377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f8156c46-709f-4aa8-b010-52e8d85b69a2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:55.477822 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f8156c46-709f-4aa8-b010-52e8d85b69a2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:55.478239 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f8156c46-709f-4aa8-b010-52e8d85b69a2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:55.610182 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f8156c46-709f-4aa8-b010-52e8d85b69a2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:55.618386 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f8156c46-709f-4aa8-b010-52e8d85b69a2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1319 microversion: 2.1 time: 0.153460 Oct 28 09:48:55.618965 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 268/552] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:55 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1319 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:56.642413 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-31bcc787-2028-46bd-8ce2-91a204c0822d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:56.643236 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-31bcc787-2028-46bd-8ce2-91a204c0822d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:56.649421 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-31bcc787-2028-46bd-8ce2-91a204c0822d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:56.649906 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-31bcc787-2028-46bd-8ce2-91a204c0822d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:56.650331 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-31bcc787-2028-46bd-8ce2-91a204c0822d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:56.651040 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-31bcc787-2028-46bd-8ce2-91a204c0822d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:56.651496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-31bcc787-2028-46bd-8ce2-91a204c0822d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:56.651891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-31bcc787-2028-46bd-8ce2-91a204c0822d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:56.844491 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-31bcc787-2028-46bd-8ce2-91a204c0822d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:56.852758 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-31bcc787-2028-46bd-8ce2-91a204c0822d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1319 microversion: 2.1 time: 0.214999 Oct 28 09:48:56.853544 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 285/553] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:56 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1319 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:57.875613 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9582ff45-8cbb-48b5-b3e6-729767fdc423 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:57.876242 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9582ff45-8cbb-48b5-b3e6-729767fdc423 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:57.883243 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9582ff45-8cbb-48b5-b3e6-729767fdc423 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:57.883712 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9582ff45-8cbb-48b5-b3e6-729767fdc423 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:57.884135 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9582ff45-8cbb-48b5-b3e6-729767fdc423 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:57.884807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9582ff45-8cbb-48b5-b3e6-729767fdc423 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:57.885246 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9582ff45-8cbb-48b5-b3e6-729767fdc423 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:57.885654 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9582ff45-8cbb-48b5-b3e6-729767fdc423 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:58.020690 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-9582ff45-8cbb-48b5-b3e6-729767fdc423 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:58.028382 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9582ff45-8cbb-48b5-b3e6-729767fdc423 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1319 microversion: 2.1 time: 0.156476 Oct 28 09:48:58.028942 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 269/554] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:57 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:48:59.052179 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d2122897-5301-46c1-9fe9-38bb022bb7ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:48:59.052811 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d2122897-5301-46c1-9fe9-38bb022bb7ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:48:59.059155 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d2122897-5301-46c1-9fe9-38bb022bb7ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:59.059640 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d2122897-5301-46c1-9fe9-38bb022bb7ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:59.060076 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d2122897-5301-46c1-9fe9-38bb022bb7ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:59.060767 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d2122897-5301-46c1-9fe9-38bb022bb7ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:48:59.061347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d2122897-5301-46c1-9fe9-38bb022bb7ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:48:59.061628 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d2122897-5301-46c1-9fe9-38bb022bb7ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:48:59.196115 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d2122897-5301-46c1-9fe9-38bb022bb7ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:48:59.203755 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d2122897-5301-46c1-9fe9-38bb022bb7ec tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1319 microversion: 2.1 time: 0.156011 Oct 28 09:48:59.204710 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 286/555] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:48:59 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1319 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:00.226772 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1ee25238-4005-42ca-ac4b-1fc4b55e145b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:00.227308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1ee25238-4005-42ca-ac4b-1fc4b55e145b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:00.233719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ee25238-4005-42ca-ac4b-1fc4b55e145b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:00.234197 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ee25238-4005-42ca-ac4b-1fc4b55e145b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:00.234656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ee25238-4005-42ca-ac4b-1fc4b55e145b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:00.235338 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ee25238-4005-42ca-ac4b-1fc4b55e145b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:00.235785 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ee25238-4005-42ca-ac4b-1fc4b55e145b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:00.236199 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ee25238-4005-42ca-ac4b-1fc4b55e145b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:00.240147 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=100 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:00.368367 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1ee25238-4005-42ca-ac4b-1fc4b55e145b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:00.376081 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1ee25238-4005-42ca-ac4b-1fc4b55e145b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1319 microversion: 2.1 time: 0.153245 Oct 28 09:49:00.376644 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 270/556] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:00 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1319 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:01.135325 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-82c1e848-5fc9-4360-b89a-46a9f0c8861b req-3a34baa2-f675-4714-9806-d52b13dba9b8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "85327eca-6324-41a4-8b03-f2d5226cf2f5", "name": "network-vif-plugged", "status": "completed", "tag": "be115348-3000-4d12-9056-f68416196802"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:01.147558 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-82c1e848-5fc9-4360-b89a-46a9f0c8861b req-3a34baa2-f675-4714-9806-d52b13dba9b8 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:01.148067 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-82c1e848-5fc9-4360-b89a-46a9f0c8861b req-3a34baa2-f675-4714-9806-d52b13dba9b8 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:01.148483 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-82c1e848-5fc9-4360-b89a-46a9f0c8861b req-3a34baa2-f675-4714-9806-d52b13dba9b8 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:01.160359 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-82c1e848-5fc9-4360-b89a-46a9f0c8861b req-3a34baa2-f675-4714-9806-d52b13dba9b8 service nova] Creating event network-vif-plugged:be115348-3000-4d12-9056-f68416196802 for instance 85327eca-6324-41a4-8b03-f2d5226cf2f5 on hv-cinder-zed-3 Oct 28 09:49:01.166301 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-82c1e848-5fc9-4360-b89a-46a9f0c8861b req-3a34baa2-f675-4714-9806-d52b13dba9b8 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034509 Oct 28 09:49:01.167187 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 287/557] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:49:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:01.399235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f803c36c-3a6a-47a5-a8b4-20c854afe46c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:01.399834 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f803c36c-3a6a-47a5-a8b4-20c854afe46c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:01.406170 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f803c36c-3a6a-47a5-a8b4-20c854afe46c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:01.406676 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f803c36c-3a6a-47a5-a8b4-20c854afe46c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:01.407106 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f803c36c-3a6a-47a5-a8b4-20c854afe46c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:01.407784 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f803c36c-3a6a-47a5-a8b4-20c854afe46c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:01.408233 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f803c36c-3a6a-47a5-a8b4-20c854afe46c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:01.408646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f803c36c-3a6a-47a5-a8b4-20c854afe46c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:01.557581 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f803c36c-3a6a-47a5-a8b4-20c854afe46c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:01.566485 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f803c36c-3a6a-47a5-a8b4-20c854afe46c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1319 microversion: 2.1 time: 0.171020 Oct 28 09:49:01.567048 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 271/558] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:01 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1319 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:01.584188 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-4593fd2e-5ce7-44d4-ab1c-7f0ae89ed136 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "855268e6-c6f2-4ab9-a6e6-79c92b1e0b32"}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:01.586664 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-4593fd2e-5ce7-44d4-ab1c-7f0ae89ed136 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:01.592965 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4593fd2e-5ce7-44d4-ab1c-7f0ae89ed136 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:01.593450 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4593fd2e-5ce7-44d4-ab1c-7f0ae89ed136 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:01.593884 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4593fd2e-5ce7-44d4-ab1c-7f0ae89ed136 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:01.594632 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4593fd2e-5ce7-44d4-ab1c-7f0ae89ed136 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:01.595086 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4593fd2e-5ce7-44d4-ab1c-7f0ae89ed136 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:01.595510 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4593fd2e-5ce7-44d4-ab1c-7f0ae89ed136 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:01.930416 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=2 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:01.951294 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=6 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:02.590630 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-95e148d7-0859-49af-9785-f28531118099 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:02.591464 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-95e148d7-0859-49af-9785-f28531118099 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:02.593560 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:02.606602 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95e148d7-0859-49af-9785-f28531118099 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:02.607170 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95e148d7-0859-49af-9785-f28531118099 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:02.607600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95e148d7-0859-49af-9785-f28531118099 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:02.608284 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95e148d7-0859-49af-9785-f28531118099 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:02.608725 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95e148d7-0859-49af-9785-f28531118099 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:02.609137 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95e148d7-0859-49af-9785-f28531118099 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:02.655003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-4593fd2e-5ce7-44d4-ab1c-7f0ae89ed136 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "POST /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.075370 Oct 28 09:49:02.660481 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 288/559] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:49:01 2022] POST /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-volume_attachments => generated 194 bytes in 1077 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:02.751948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-95e148d7-0859-49af-9785-f28531118099 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:02.759624 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-95e148d7-0859-49af-9785-f28531118099 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1319 microversion: 2.1 time: 0.173624 Oct 28 09:49:02.760197 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 272/560] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:02 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1319 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:03.550712 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-532fd61d-7c46-4095-9218-7870cd51b8d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:03.550712 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-532fd61d-7c46-4095-9218-7870cd51b8d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:03.557533 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-532fd61d-7c46-4095-9218-7870cd51b8d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:03.558019 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-532fd61d-7c46-4095-9218-7870cd51b8d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:03.558484 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-532fd61d-7c46-4095-9218-7870cd51b8d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:03.559183 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-532fd61d-7c46-4095-9218-7870cd51b8d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:03.559630 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-532fd61d-7c46-4095-9218-7870cd51b8d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:03.560053 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-532fd61d-7c46-4095-9218-7870cd51b8d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:03.686721 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.objects.instance [None req-532fd61d-7c46-4095-9218-7870cd51b8d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lazy-loading 'services' on Instance uuid b981eca6-4d6f-4e48-bf62-016833f2885d {{(pid=127029) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:49:03.721565 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:03.782968 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-59a3408b-02a8-4067-b78e-d1dfdc11e07d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:03.784085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-59a3408b-02a8-4067-b78e-d1dfdc11e07d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:03.790805 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a3408b-02a8-4067-b78e-d1dfdc11e07d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:03.791287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a3408b-02a8-4067-b78e-d1dfdc11e07d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:03.791727 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a3408b-02a8-4067-b78e-d1dfdc11e07d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:03.792415 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a3408b-02a8-4067-b78e-d1dfdc11e07d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:03.792858 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a3408b-02a8-4067-b78e-d1dfdc11e07d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:03.793273 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a3408b-02a8-4067-b78e-d1dfdc11e07d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:03.823619 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-532fd61d-7c46-4095-9218-7870cd51b8d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "DELETE /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/os-volume_attachments/505290f3-07f2-4f85-8d41-60f298323282" status: 202 len: 0 microversion: 2.1 time: 0.277726 Oct 28 09:49:03.824239 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 289/561] 10.222.0.25 () {58 vars in 1416 bytes} [Fri Oct 28 09:49:03 2022] DELETE /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/os-volume_attachments/505290f3-07f2-4f85-8d41-60f298323282 => generated 0 bytes in 279 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:49:03.949218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-59a3408b-02a8-4067-b78e-d1dfdc11e07d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:03.958152 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-59a3408b-02a8-4067-b78e-d1dfdc11e07d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1509 microversion: 2.1 time: 0.178874 Oct 28 09:49:03.958832 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 273/562] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:03 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1509 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:03.982097 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-046dc033-7b22-4596-bd7f-780a2e62158e tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f61bb8b5-c2d8-4319-b39b-25de29ecd6bc", "device": "/dev/sdb"}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:03.985118 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-046dc033-7b22-4596-bd7f-780a2e62158e tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:03.992242 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-046dc033-7b22-4596-bd7f-780a2e62158e tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:03.992668 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-046dc033-7b22-4596-bd7f-780a2e62158e tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:03.993113 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-046dc033-7b22-4596-bd7f-780a2e62158e tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:03.993816 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-046dc033-7b22-4596-bd7f-780a2e62158e tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:03.994447 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-046dc033-7b22-4596-bd7f-780a2e62158e tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:03.994719 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-046dc033-7b22-4596-bd7f-780a2e62158e tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:04.294236 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-046dc033-7b22-4596-bd7f-780a2e62158e tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "POST /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.316516 Oct 28 09:49:04.295083 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 290/563] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:49:03 2022] POST /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5/os-volume_attachments => generated 194 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:11.270875 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-dc24de49-6fad-4eb5-bb55-3f5a1d02b508 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:11.273181 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-dc24de49-6fad-4eb5-bb55-3f5a1d02b508 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:11.279865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dc24de49-6fad-4eb5-bb55-3f5a1d02b508 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:11.280349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dc24de49-6fad-4eb5-bb55-3f5a1d02b508 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:11.280776 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dc24de49-6fad-4eb5-bb55-3f5a1d02b508 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:11.281567 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dc24de49-6fad-4eb5-bb55-3f5a1d02b508 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:11.282048 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dc24de49-6fad-4eb5-bb55-3f5a1d02b508 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:11.282499 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dc24de49-6fad-4eb5-bb55-3f5a1d02b508 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:11.306485 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-dc24de49-6fad-4eb5-bb55-3f5a1d02b508 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lazy-loading 'flavor' on Instance uuid 4891b63b-eb08-448d-a640-f9579c8140ff {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:49:11.322151 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8ae65bae-8141-474d-81d4-594ae69ece7e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:11.328893 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8ae65bae-8141-474d-81d4-594ae69ece7e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:11.329376 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8ae65bae-8141-474d-81d4-594ae69ece7e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:11.329811 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8ae65bae-8141-474d-81d4-594ae69ece7e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:11.397688 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8ae65bae-8141-474d-81d4-594ae69ece7e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Going to try to stop instance {{(pid=127029) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 28 09:49:11.437700 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-dc24de49-6fad-4eb5-bb55-3f5a1d02b508 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "POST /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/action" status: 200 len: 32 microversion: 2.1 time: 0.171146 Oct 28 09:49:11.443541 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 274/564] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:49:11 2022] POST /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/action => generated 32 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 09:49:11.447005 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8ae65bae-8141-474d-81d4-594ae69ece7e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "POST /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/action" status: 202 len: 0 microversion: 2.1 time: 0.129020 Oct 28 09:49:11.447957 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 291/565] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:49:11 2022] POST /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/action => generated 0 bytes in 131 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 09:49:11.449668 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-605da71a-661b-48ee-9d44-9cf054f72ef9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:11.450633 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-605da71a-661b-48ee-9d44-9cf054f72ef9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:11.457452 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-605da71a-661b-48ee-9d44-9cf054f72ef9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:11.458003 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-9ce2fd37-6d22-42cd-aab3-97c1139a6e40 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:11.458174 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-605da71a-661b-48ee-9d44-9cf054f72ef9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:11.458684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-605da71a-661b-48ee-9d44-9cf054f72ef9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:11.459285 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-9ce2fd37-6d22-42cd-aab3-97c1139a6e40 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:11.459860 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-605da71a-661b-48ee-9d44-9cf054f72ef9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:11.460510 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-605da71a-661b-48ee-9d44-9cf054f72ef9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:11.461097 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-605da71a-661b-48ee-9d44-9cf054f72ef9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:11.468047 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9ce2fd37-6d22-42cd-aab3-97c1139a6e40 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:11.468526 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9ce2fd37-6d22-42cd-aab3-97c1139a6e40 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:11.468961 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9ce2fd37-6d22-42cd-aab3-97c1139a6e40 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:11.469686 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9ce2fd37-6d22-42cd-aab3-97c1139a6e40 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:11.470130 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9ce2fd37-6d22-42cd-aab3-97c1139a6e40 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:11.470628 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9ce2fd37-6d22-42cd-aab3-97c1139a6e40 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:11.705713 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-605da71a-661b-48ee-9d44-9cf054f72ef9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:11.714361 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-605da71a-661b-48ee-9d44-9cf054f72ef9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.269264 Oct 28 09:49:11.723180 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-9ce2fd37-6d22-42cd-aab3-97c1139a6e40 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:11.727292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 275/566] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:11 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:11.732010 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-9ce2fd37-6d22-42cd-aab3-97c1139a6e40 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.279666 Oct 28 09:49:11.732640 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 292/567] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:11 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:11.757251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6dc23de9-2b9d-4bf2-bf17-ecd5d92f7f44 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:11.757643 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6dc23de9-2b9d-4bf2-bf17-ecd5d92f7f44 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:11.763938 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6dc23de9-2b9d-4bf2-bf17-ecd5d92f7f44 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:11.764431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6dc23de9-2b9d-4bf2-bf17-ecd5d92f7f44 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:11.764861 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6dc23de9-2b9d-4bf2-bf17-ecd5d92f7f44 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:11.765547 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6dc23de9-2b9d-4bf2-bf17-ecd5d92f7f44 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:11.765990 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6dc23de9-2b9d-4bf2-bf17-ecd5d92f7f44 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:11.766424 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6dc23de9-2b9d-4bf2-bf17-ecd5d92f7f44 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:11.791387 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6dc23de9-2b9d-4bf2-bf17-ecd5d92f7f44 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b/ips" status: 200 len: 150 microversion: 2.1 time: 0.037927 Oct 28 09:49:11.791979 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 276/568] 10.222.0.25 () {58 vars in 1248 bytes} [Fri Oct 28 09:49:11 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b/ips => generated 150 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:11.805310 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5f5aaf1d-5aa2-4dae-84cd-79ebba97104e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d2d16afe-439c-43aa-9b38-91658d3478cd"}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:11.807984 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5f5aaf1d-5aa2-4dae-84cd-79ebba97104e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:11.813960 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5f5aaf1d-5aa2-4dae-84cd-79ebba97104e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:11.814963 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5f5aaf1d-5aa2-4dae-84cd-79ebba97104e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:11.815392 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5f5aaf1d-5aa2-4dae-84cd-79ebba97104e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:11.816074 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5f5aaf1d-5aa2-4dae-84cd-79ebba97104e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:11.816520 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5f5aaf1d-5aa2-4dae-84cd-79ebba97104e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:11.817690 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5f5aaf1d-5aa2-4dae-84cd-79ebba97104e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:11.909572 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.wsgi [None req-5f5aaf1d-5aa2-4dae-84cd-79ebba97104e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Invalid volume: volume d2d16afe-439c-43aa-9b38-91658d3478cd is already attached to instances: 82790f9d-85c2-4205-b9ff-290df00c2edf Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5f5aaf1d-5aa2-4dae-84cd-79ebba97104e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: request: POST /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b/os-volume_attachments HTTP/1.1 Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept: application/json Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept-Encoding: identity Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Connection: close Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Length: 74 Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Type: application/json Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Host: 10.222.0.25 Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Openstack-System-Scope: None Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: User-Agent: python-urllib3/1.26.12 Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Id: None Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Name: None Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Identity-Status: Confirmed Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Is-Admin-Project: True Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Id: default Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Name: Default Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Role: member,reader Oct 28 09:49:11.911341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Roles: member,reader Oct 28 09:49:11.912771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:49:11.912771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:49:11.912771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:49:11.912771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:49:11.912771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:49:11.912771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Id: default Oct 28 09:49:11.912771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Name: Default Oct 28 09:49:11.912771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:49:11.912771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:49:11.912771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Oct 28 09:49:11.912771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: {"volumeAttachment": {"volumeId": "d2d16afe-439c-43aa-9b38-91658d3478cd"}} Oct 28 09:49:11.912771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: method: > Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: exception: Invalid volume: volume d2d16afe-439c-43aa-9b38-91658d3478cd is already attached to instances: 82790f9d-85c2-4205-b9ff-290df00c2edf Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: action_args: {'server_id': 'f8edd976-e011-4194-ad94-1160cc01c63b', 'body': {'volumeAttachment': {'volumeId': 'd2d16afe-439c-43aa-9b38-91658d3478cd'}}} Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume d2d16afe-439c-43aa-9b38-91658d3478cd is already attached to instances: 82790f9d-85c2-4205-b9ff-290df00c2edf Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:11.913552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume d2d16afe-439c-43aa-9b38-91658d3478cd is already attached to instances: 82790f9d-85c2-4205-b9ff-290df00c2edf Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:11.914878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:49:11.916347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:49:11.916347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:49:11.916347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:49:11.916347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume d2d16afe-439c-43aa-9b38-91658d3478cd is already attached to instances: 82790f9d-85c2-4205-b9ff-290df00c2edf Oct 28 09:49:11.916347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:11.916347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5f5aaf1d-5aa2-4dae-84cd-79ebba97104e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 400 to user: Invalid volume: volume d2d16afe-439c-43aa-9b38-91658d3478cd is already attached to instances: 82790f9d-85c2-4205-b9ff-290df00c2edf {{(pid=127029) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:49:11.916347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5f5aaf1d-5aa2-4dae-84cd-79ebba97104e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "POST /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.111715 Oct 28 09:49:11.916347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 293/569] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:49:11 2022] POST /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b/os-volume_attachments => generated 174 bytes in 113 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 28 09:49:11.922695 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-acd156aa-483e-4080-951f-33597508869b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:11.923191 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-acd156aa-483e-4080-951f-33597508869b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:11.929947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-acd156aa-483e-4080-951f-33597508869b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:11.930480 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-acd156aa-483e-4080-951f-33597508869b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:11.930917 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-acd156aa-483e-4080-951f-33597508869b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:11.931647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-acd156aa-483e-4080-951f-33597508869b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:11.932128 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-acd156aa-483e-4080-951f-33597508869b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:11.932551 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-acd156aa-483e-4080-951f-33597508869b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:11.965926 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-acd156aa-483e-4080-951f-33597508869b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Going to try to terminate instance {{(pid=127030) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:49:12.048911 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-acd156aa-483e-4080-951f-33597508869b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 204 len: 0 microversion: 2.1 time: 0.130329 Oct 28 09:49:12.049722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 277/570] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:49:11 2022] DELETE /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:49:12.058902 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6d07da53-7052-40c7-8ca7-16ec311ddc0a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:12.059645 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6d07da53-7052-40c7-8ca7-16ec311ddc0a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:12.067445 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6d07da53-7052-40c7-8ca7-16ec311ddc0a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:12.068018 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6d07da53-7052-40c7-8ca7-16ec311ddc0a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:12.068519 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6d07da53-7052-40c7-8ca7-16ec311ddc0a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:12.069283 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6d07da53-7052-40c7-8ca7-16ec311ddc0a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:12.069798 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6d07da53-7052-40c7-8ca7-16ec311ddc0a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:12.070218 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6d07da53-7052-40c7-8ca7-16ec311ddc0a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:12.203556 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6d07da53-7052-40c7-8ca7-16ec311ddc0a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:12.211475 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6d07da53-7052-40c7-8ca7-16ec311ddc0a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1682 microversion: 2.1 time: 0.156414 Oct 28 09:49:12.212033 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 294/571] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:12 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1682 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:12.739351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0bb35d4d-33c2-4de8-a2df-e0ba1b50c101 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:12.740034 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0bb35d4d-33c2-4de8-a2df-e0ba1b50c101 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:12.746443 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bb35d4d-33c2-4de8-a2df-e0ba1b50c101 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:12.746931 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bb35d4d-33c2-4de8-a2df-e0ba1b50c101 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:12.747360 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bb35d4d-33c2-4de8-a2df-e0ba1b50c101 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:12.748037 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bb35d4d-33c2-4de8-a2df-e0ba1b50c101 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:12.748576 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bb35d4d-33c2-4de8-a2df-e0ba1b50c101 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:12.749001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bb35d4d-33c2-4de8-a2df-e0ba1b50c101 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:12.757167 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-0d548d30-9fb3-4b21-a02b-329636d8528d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:12.757916 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-0d548d30-9fb3-4b21-a02b-329636d8528d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:12.764335 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d548d30-9fb3-4b21-a02b-329636d8528d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:12.764813 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d548d30-9fb3-4b21-a02b-329636d8528d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:12.765240 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d548d30-9fb3-4b21-a02b-329636d8528d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:12.765932 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d548d30-9fb3-4b21-a02b-329636d8528d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:12.766380 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d548d30-9fb3-4b21-a02b-329636d8528d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:12.766831 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d548d30-9fb3-4b21-a02b-329636d8528d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:12.941850 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0bb35d4d-33c2-4de8-a2df-e0ba1b50c101 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:12.950435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0bb35d4d-33c2-4de8-a2df-e0ba1b50c101 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.215991 Oct 28 09:49:12.951017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 278/572] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:12 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:12.953069 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0d548d30-9fb3-4b21-a02b-329636d8528d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:12.961351 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-0d548d30-9fb3-4b21-a02b-329636d8528d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.207989 Oct 28 09:49:12.961935 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 295/573] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:12 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:13.234507 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7f56f5af-845d-4d31-a420-987f5110b3bd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:13.235061 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7f56f5af-845d-4d31-a420-987f5110b3bd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:13.241388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7f56f5af-845d-4d31-a420-987f5110b3bd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:13.241873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7f56f5af-845d-4d31-a420-987f5110b3bd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:13.242309 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7f56f5af-845d-4d31-a420-987f5110b3bd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:13.243038 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7f56f5af-845d-4d31-a420-987f5110b3bd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:13.243511 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7f56f5af-845d-4d31-a420-987f5110b3bd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:13.243917 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7f56f5af-845d-4d31-a420-987f5110b3bd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:13.378096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7f56f5af-845d-4d31-a420-987f5110b3bd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:13.385796 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7f56f5af-845d-4d31-a420-987f5110b3bd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1682 microversion: 2.1 time: 0.155338 Oct 28 09:49:13.386358 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 279/574] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:13 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1682 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:13.973306 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8f3d51e7-e148-47af-a37a-fff8841257f3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:13.973898 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8f3d51e7-e148-47af-a37a-fff8841257f3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:13.976419 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=53,nova_api:INSERT=3 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:13.982561 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8f3d51e7-e148-47af-a37a-fff8841257f3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:13.983051 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8f3d51e7-e148-47af-a37a-fff8841257f3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:13.983484 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8f3d51e7-e148-47af-a37a-fff8841257f3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:13.984647 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8f3d51e7-e148-47af-a37a-fff8841257f3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:13.984935 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-16f4004a-705d-45f8-a090-99edbd3e22ad tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:13.985394 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8f3d51e7-e148-47af-a37a-fff8841257f3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:13.985688 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8f3d51e7-e148-47af-a37a-fff8841257f3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:13.986140 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-16f4004a-705d-45f8-a090-99edbd3e22ad tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:13.993020 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-16f4004a-705d-45f8-a090-99edbd3e22ad tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:13.993780 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-16f4004a-705d-45f8-a090-99edbd3e22ad tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:13.994208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-16f4004a-705d-45f8-a090-99edbd3e22ad tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:13.994931 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-16f4004a-705d-45f8-a090-99edbd3e22ad tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:13.995383 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-16f4004a-705d-45f8-a090-99edbd3e22ad tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:13.995790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-16f4004a-705d-45f8-a090-99edbd3e22ad tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:14.189878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-8f3d51e7-e148-47af-a37a-fff8841257f3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:14.193044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-16f4004a-705d-45f8-a090-99edbd3e22ad tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:14.198518 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8f3d51e7-e148-47af-a37a-fff8841257f3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.229224 Oct 28 09:49:14.199106 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 296/575] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:13 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:14.201281 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-16f4004a-705d-45f8-a090-99edbd3e22ad tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.220043 Oct 28 09:49:14.201857 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 280/576] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:13 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:14.410571 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-820b9955-c747-44f3-803b-3a76ec900ad2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:14.411101 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-820b9955-c747-44f3-803b-3a76ec900ad2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:14.418070 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-820b9955-c747-44f3-803b-3a76ec900ad2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:14.418593 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-820b9955-c747-44f3-803b-3a76ec900ad2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:14.419419 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-820b9955-c747-44f3-803b-3a76ec900ad2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:14.420151 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-820b9955-c747-44f3-803b-3a76ec900ad2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:14.420630 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-820b9955-c747-44f3-803b-3a76ec900ad2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:14.421050 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-820b9955-c747-44f3-803b-3a76ec900ad2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:14.694477 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-820b9955-c747-44f3-803b-3a76ec900ad2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:14.702282 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-820b9955-c747-44f3-803b-3a76ec900ad2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1682 microversion: 2.1 time: 0.295613 Oct 28 09:49:14.702887 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 297/577] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:14 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1682 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:15.223810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-80a15c02-3c5d-4312-a133-c7b472acaebc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:15.224738 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-80a15c02-3c5d-4312-a133-c7b472acaebc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:15.225885 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-0b2ad93f-45bd-41a7-a403-934adb140a23 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:15.226834 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-0b2ad93f-45bd-41a7-a403-934adb140a23 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:15.234232 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0b2ad93f-45bd-41a7-a403-934adb140a23 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:15.234780 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0b2ad93f-45bd-41a7-a403-934adb140a23 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:15.235513 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0b2ad93f-45bd-41a7-a403-934adb140a23 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:15.235800 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-80a15c02-3c5d-4312-a133-c7b472acaebc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:15.236286 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0b2ad93f-45bd-41a7-a403-934adb140a23 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:15.236396 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-80a15c02-3c5d-4312-a133-c7b472acaebc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:15.236745 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0b2ad93f-45bd-41a7-a403-934adb140a23 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:15.237163 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0b2ad93f-45bd-41a7-a403-934adb140a23 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:15.239998 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-80a15c02-3c5d-4312-a133-c7b472acaebc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:15.240800 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-80a15c02-3c5d-4312-a133-c7b472acaebc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:15.241310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-80a15c02-3c5d-4312-a133-c7b472acaebc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:15.241739 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-80a15c02-3c5d-4312-a133-c7b472acaebc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:15.516958 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-80a15c02-3c5d-4312-a133-c7b472acaebc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:15.521299 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0b2ad93f-45bd-41a7-a403-934adb140a23 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:15.528701 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-80a15c02-3c5d-4312-a133-c7b472acaebc tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.309450 Oct 28 09:49:15.529988 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-0b2ad93f-45bd-41a7-a403-934adb140a23 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.308146 Oct 28 09:49:15.531081 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 298/578] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:15 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:15.531451 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 281/579] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:15 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:15.536199 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-128e5e7b-0d37-41a6-90f2-f6df52280d96 req-c7860b09-5c91-4051-b13f-de8e58ce04ef service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f8edd976-e011-4194-ad94-1160cc01c63b", "name": "network-vif-deleted", "tag": "450bfe2d-019d-4361-97e8-cd9a4452887d"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:15.547253 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-128e5e7b-0d37-41a6-90f2-f6df52280d96 req-c7860b09-5c91-4051-b13f-de8e58ce04ef service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:15.547984 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-128e5e7b-0d37-41a6-90f2-f6df52280d96 req-c7860b09-5c91-4051-b13f-de8e58ce04ef service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:15.548647 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-128e5e7b-0d37-41a6-90f2-f6df52280d96 req-c7860b09-5c91-4051-b13f-de8e58ce04ef service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:15.569350 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-128e5e7b-0d37-41a6-90f2-f6df52280d96 req-c7860b09-5c91-4051-b13f-de8e58ce04ef service nova] Creating event network-vif-deleted:450bfe2d-019d-4361-97e8-cd9a4452887d for instance f8edd976-e011-4194-ad94-1160cc01c63b on hv-cinder-zed-3 Oct 28 09:49:15.576932 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-128e5e7b-0d37-41a6-90f2-f6df52280d96 req-c7860b09-5c91-4051-b13f-de8e58ce04ef service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043976 Oct 28 09:49:15.577744 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 299/580] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:49:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:15.725155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:15.725980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:15.732501 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:15.733203 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:15.733798 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:15.734690 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:15.735285 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:15.735857 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:15.770081 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:15.770597 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:15.771041 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:15.771494 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lazy-loading 'fault' on Instance uuid f8edd976-e011-4194-ad94-1160cc01c63b {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:49:15.817791 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:15.825734 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-03e518e8-bc8d-403a-a6da-4f872b4ba0c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 200 len: 1331 microversion: 2.1 time: 0.104425 Oct 28 09:49:15.826300 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 282/581] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:15 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 1331 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:16.559224 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-34018e2f-7a31-429c-8ee4-d26a0c076131 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:16.560040 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-34018e2f-7a31-429c-8ee4-d26a0c076131 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:16.567251 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-34018e2f-7a31-429c-8ee4-d26a0c076131 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:16.567855 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-34018e2f-7a31-429c-8ee4-d26a0c076131 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:16.568715 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-34018e2f-7a31-429c-8ee4-d26a0c076131 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:16.575360 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-48f737df-a2b6-4733-9360-b889ba252dba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:16.576722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-48f737df-a2b6-4733-9360-b889ba252dba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:16.576939 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-34018e2f-7a31-429c-8ee4-d26a0c076131 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:16.577592 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-34018e2f-7a31-429c-8ee4-d26a0c076131 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:16.578116 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-34018e2f-7a31-429c-8ee4-d26a0c076131 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:16.586490 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-48f737df-a2b6-4733-9360-b889ba252dba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:16.587234 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-48f737df-a2b6-4733-9360-b889ba252dba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:16.587910 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-48f737df-a2b6-4733-9360-b889ba252dba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:16.588833 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-48f737df-a2b6-4733-9360-b889ba252dba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:16.589440 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-48f737df-a2b6-4733-9360-b889ba252dba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:16.590039 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-48f737df-a2b6-4733-9360-b889ba252dba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:16.597089 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:16.777748 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-34018e2f-7a31-429c-8ee4-d26a0c076131 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:16.786779 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-34018e2f-7a31-429c-8ee4-d26a0c076131 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.237142 Oct 28 09:49:16.787372 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 300/582] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:16 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:16.801424 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-48f737df-a2b6-4733-9360-b889ba252dba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:16.810747 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-48f737df-a2b6-4733-9360-b889ba252dba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.240326 Oct 28 09:49:16.811550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 283/583] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:16 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:16.851291 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e77c19bd-c63d-4812-98ba-32e612a0b0c7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:16.851882 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e77c19bd-c63d-4812-98ba-32e612a0b0c7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:16.857836 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e77c19bd-c63d-4812-98ba-32e612a0b0c7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:16.858308 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e77c19bd-c63d-4812-98ba-32e612a0b0c7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:16.858767 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e77c19bd-c63d-4812-98ba-32e612a0b0c7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:16.859452 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e77c19bd-c63d-4812-98ba-32e612a0b0c7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:16.859896 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e77c19bd-c63d-4812-98ba-32e612a0b0c7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:16.860316 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e77c19bd-c63d-4812-98ba-32e612a0b0c7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:16.873623 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.wsgi [None req-e77c19bd-c63d-4812-98ba-32e612a0b0c7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e77c19bd-c63d-4812-98ba-32e612a0b0c7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: request: GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b HTTP/1.1 Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept: application/json Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept-Encoding: identity Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Connection: close Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Length: 0 Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Type: application/json Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Host: 10.222.0.25 Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Openstack-System-Scope: None Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: User-Agent: python-urllib3/1.26.12 Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Id: None Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Name: None Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Identity-Status: Confirmed Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Is-Admin-Project: True Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Id: default Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Name: Default Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Role: member,reader Oct 28 09:49:16.875323 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Roles: member,reader Oct 28 09:49:16.876928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:49:16.876928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:49:16.876928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:49:16.876928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:49:16.876928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:49:16.876928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Id: default Oct 28 09:49:16.876928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Name: Default Oct 28 09:49:16.876928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:49:16.876928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:49:16.876928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: method: > Oct 28 09:49:16.876928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: exception: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:49:16.876928 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: action_args: {'id': 'f8edd976-e011-4194-ad94-1160cc01c63b'} Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:49:16.877713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:49:16.878912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:16.880011 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e77c19bd-c63d-4812-98ba-32e612a0b0c7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 404 to user: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. {{(pid=127029) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:49:16.880011 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e77c19bd-c63d-4812-98ba-32e612a0b0c7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 404 len: 111 microversion: 2.1 time: 0.029328 Oct 28 09:49:16.880011 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 301/584] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:16 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:49:16.888257 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9f9cc5bb-8385-44e9-ad03-9082b1549c36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '.version_select at 0x7f271a456160>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:16.919351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9f9cc5bb-8385-44e9-ad03-9082b1549c36 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-624967812" status: 202 len: 0 microversion: 2.1 time: 0.037168 Oct 28 09:49:16.919923 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 284/585] 10.222.0.25 () {58 vars in 1218 bytes} [Fri Oct 28 09:49:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-624967812 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:49:17.810815 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d81b1275-149b-4219-8a01-2dfcc46c9911 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:17.811426 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d81b1275-149b-4219-8a01-2dfcc46c9911 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:17.818086 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d81b1275-149b-4219-8a01-2dfcc46c9911 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:17.818594 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d81b1275-149b-4219-8a01-2dfcc46c9911 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:17.819070 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d81b1275-149b-4219-8a01-2dfcc46c9911 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:17.819764 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d81b1275-149b-4219-8a01-2dfcc46c9911 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:17.820211 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d81b1275-149b-4219-8a01-2dfcc46c9911 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:17.820627 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d81b1275-149b-4219-8a01-2dfcc46c9911 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:17.835773 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-34783dc1-4e66-4bfe-8346-881d71d2477f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:17.836363 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-34783dc1-4e66-4bfe-8346-881d71d2477f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:17.838572 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=58,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:17.846671 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-34783dc1-4e66-4bfe-8346-881d71d2477f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:17.847745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-34783dc1-4e66-4bfe-8346-881d71d2477f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:17.848185 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-34783dc1-4e66-4bfe-8346-881d71d2477f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:17.849195 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-34783dc1-4e66-4bfe-8346-881d71d2477f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:17.849316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-34783dc1-4e66-4bfe-8346-881d71d2477f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:17.849728 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-34783dc1-4e66-4bfe-8346-881d71d2477f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:18.042426 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d81b1275-149b-4219-8a01-2dfcc46c9911 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:18.046652 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-34783dc1-4e66-4bfe-8346-881d71d2477f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:18.053005 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d81b1275-149b-4219-8a01-2dfcc46c9911 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.245761 Oct 28 09:49:18.054168 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-34783dc1-4e66-4bfe-8346-881d71d2477f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.222728 Oct 28 09:49:18.054769 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 285/586] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:17 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:18.060421 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 302/587] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:17 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:18.060902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-979f9702-20df-4fc5-b012-3811ceca7db1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:18.061388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-979f9702-20df-4fc5-b012-3811ceca7db1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:18.068012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-979f9702-20df-4fc5-b012-3811ceca7db1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:18.068702 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-979f9702-20df-4fc5-b012-3811ceca7db1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:18.069283 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-979f9702-20df-4fc5-b012-3811ceca7db1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:18.070143 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-979f9702-20df-4fc5-b012-3811ceca7db1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:18.070888 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-979f9702-20df-4fc5-b012-3811ceca7db1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:18.071266 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-979f9702-20df-4fc5-b012-3811ceca7db1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:18.169521 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-979f9702-20df-4fc5-b012-3811ceca7db1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lazy-loading 'services' on Instance uuid 82790f9d-85c2-4205-b9ff-290df00c2edf {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:49:18.279289 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-979f9702-20df-4fc5-b012-3811ceca7db1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments/d2d16afe-439c-43aa-9b38-91658d3478cd" status: 202 len: 0 microversion: 2.1 time: 0.221948 Oct 28 09:49:18.279824 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 286/588] 10.222.0.25 () {58 vars in 1416 bytes} [Fri Oct 28 09:49:18 2022] DELETE /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments/d2d16afe-439c-43aa-9b38-91658d3478cd => generated 0 bytes in 223 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:49:18.291256 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-15bba727-2ef7-4c3e-880b-87b942cbfaa0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:18.294929 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-15bba727-2ef7-4c3e-880b-87b942cbfaa0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:18.305502 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-15bba727-2ef7-4c3e-880b-87b942cbfaa0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:18.306334 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-15bba727-2ef7-4c3e-880b-87b942cbfaa0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:18.307129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-15bba727-2ef7-4c3e-880b-87b942cbfaa0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:18.308215 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-15bba727-2ef7-4c3e-880b-87b942cbfaa0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:18.308942 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-15bba727-2ef7-4c3e-880b-87b942cbfaa0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:18.309722 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-15bba727-2ef7-4c3e-880b-87b942cbfaa0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:18.351399 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-15bba727-2ef7-4c3e-880b-87b942cbfaa0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066082 Oct 28 09:49:18.351967 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 303/589] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:49:18 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:19.078519 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-7fdc0f26-fd2c-4a51-85be-27014374d492 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:19.079172 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e751460f-ce35-4bfc-92a6-449252c568d8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:19.079579 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-7fdc0f26-fd2c-4a51-85be-27014374d492 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:19.080422 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e751460f-ce35-4bfc-92a6-449252c568d8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:19.086456 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fdc0f26-fd2c-4a51-85be-27014374d492 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:19.086937 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fdc0f26-fd2c-4a51-85be-27014374d492 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:19.087042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e751460f-ce35-4bfc-92a6-449252c568d8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:19.087368 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fdc0f26-fd2c-4a51-85be-27014374d492 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:19.087499 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e751460f-ce35-4bfc-92a6-449252c568d8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:19.088021 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e751460f-ce35-4bfc-92a6-449252c568d8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:19.088139 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fdc0f26-fd2c-4a51-85be-27014374d492 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:19.088504 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fdc0f26-fd2c-4a51-85be-27014374d492 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:19.088872 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e751460f-ce35-4bfc-92a6-449252c568d8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:19.089045 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7fdc0f26-fd2c-4a51-85be-27014374d492 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:19.089319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e751460f-ce35-4bfc-92a6-449252c568d8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:19.089767 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e751460f-ce35-4bfc-92a6-449252c568d8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:19.298375 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e751460f-ce35-4bfc-92a6-449252c568d8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:19.304955 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7fdc0f26-fd2c-4a51-85be-27014374d492 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:19.309455 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e751460f-ce35-4bfc-92a6-449252c568d8 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.234905 Oct 28 09:49:19.310078 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 287/590] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:19 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:19.315601 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-7fdc0f26-fd2c-4a51-85be-27014374d492 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.240755 Oct 28 09:49:19.316162 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 304/591] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:19 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:19.364249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2381ceae-8118-4845-aae6-9cda1b4ad7d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:19.366374 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2381ceae-8118-4845-aae6-9cda1b4ad7d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:19.372317 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2381ceae-8118-4845-aae6-9cda1b4ad7d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:19.372784 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2381ceae-8118-4845-aae6-9cda1b4ad7d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:19.373209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2381ceae-8118-4845-aae6-9cda1b4ad7d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:19.373887 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2381ceae-8118-4845-aae6-9cda1b4ad7d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:19.374351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2381ceae-8118-4845-aae6-9cda1b4ad7d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:19.374800 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2381ceae-8118-4845-aae6-9cda1b4ad7d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:19.405087 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2381ceae-8118-4845-aae6-9cda1b4ad7d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044793 Oct 28 09:49:19.405635 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 288/592] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:49:19 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:20.343792 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-07d69572-0342-4d83-9d9d-0089496270a9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:20.344549 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-07d69572-0342-4d83-9d9d-0089496270a9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:20.347982 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e04a183b-ee6b-43f7-b578-da31526be730 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:20.348750 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e04a183b-ee6b-43f7-b578-da31526be730 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:20.352293 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-07d69572-0342-4d83-9d9d-0089496270a9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:20.352923 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-07d69572-0342-4d83-9d9d-0089496270a9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:20.353402 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-07d69572-0342-4d83-9d9d-0089496270a9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:20.354282 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-07d69572-0342-4d83-9d9d-0089496270a9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:20.354923 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-07d69572-0342-4d83-9d9d-0089496270a9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:20.355358 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-07d69572-0342-4d83-9d9d-0089496270a9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:20.355993 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e04a183b-ee6b-43f7-b578-da31526be730 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:20.356489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e04a183b-ee6b-43f7-b578-da31526be730 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:20.356926 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e04a183b-ee6b-43f7-b578-da31526be730 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:20.357653 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e04a183b-ee6b-43f7-b578-da31526be730 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:20.358111 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e04a183b-ee6b-43f7-b578-da31526be730 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:20.358566 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e04a183b-ee6b-43f7-b578-da31526be730 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:20.360500 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=3,nova_cell1:UPDATE=3 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:20.561234 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e04a183b-ee6b-43f7-b578-da31526be730 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:20.568641 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-07d69572-0342-4d83-9d9d-0089496270a9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:20.569919 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e04a183b-ee6b-43f7-b578-da31526be730 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.225686 Oct 28 09:49:20.572574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 289/593] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:20 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:20.577003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-57ed5cad-0886-4cd0-bc13-8e9136a6d269 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:20.579276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-57ed5cad-0886-4cd0-bc13-8e9136a6d269 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:20.589017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-57ed5cad-0886-4cd0-bc13-8e9136a6d269 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:20.589432 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-07d69572-0342-4d83-9d9d-0089496270a9 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.248630 Oct 28 09:49:20.589651 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-57ed5cad-0886-4cd0-bc13-8e9136a6d269 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:20.589914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-57ed5cad-0886-4cd0-bc13-8e9136a6d269 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:20.590653 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-57ed5cad-0886-4cd0-bc13-8e9136a6d269 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:20.591013 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 305/594] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:20 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:20.591204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-57ed5cad-0886-4cd0-bc13-8e9136a6d269 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:20.591517 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-57ed5cad-0886-4cd0-bc13-8e9136a6d269 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:20.624051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-57ed5cad-0886-4cd0-bc13-8e9136a6d269 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052150 Oct 28 09:49:20.624517 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 290/595] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:49:20 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:21.604172 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-288cb50c-7061-4c7c-bd49-7457b4e522a1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:21.605131 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-288cb50c-7061-4c7c-bd49-7457b4e522a1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:21.613794 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-288cb50c-7061-4c7c-bd49-7457b4e522a1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:21.614581 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-288cb50c-7061-4c7c-bd49-7457b4e522a1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:21.615293 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-288cb50c-7061-4c7c-bd49-7457b4e522a1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:21.616416 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-288cb50c-7061-4c7c-bd49-7457b4e522a1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:21.617144 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-288cb50c-7061-4c7c-bd49-7457b4e522a1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:21.617838 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-288cb50c-7061-4c7c-bd49-7457b4e522a1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:21.621696 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1698c090-f1d5-4ae4-a96b-18719efceb88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:21.622743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1698c090-f1d5-4ae4-a96b-18719efceb88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:21.632861 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1698c090-f1d5-4ae4-a96b-18719efceb88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:21.633632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1698c090-f1d5-4ae4-a96b-18719efceb88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:21.634312 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1698c090-f1d5-4ae4-a96b-18719efceb88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:21.635497 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1698c090-f1d5-4ae4-a96b-18719efceb88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:21.636236 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1698c090-f1d5-4ae4-a96b-18719efceb88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:21.636924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1698c090-f1d5-4ae4-a96b-18719efceb88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:21.838984 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-288cb50c-7061-4c7c-bd49-7457b4e522a1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:21.846860 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1698c090-f1d5-4ae4-a96b-18719efceb88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:21.847373 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-288cb50c-7061-4c7c-bd49-7457b4e522a1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.247374 Oct 28 09:49:21.848141 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 306/596] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:21 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:21.852507 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-f68878f6-9c69-432f-ac95-074064234519 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:21.854654 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-f68878f6-9c69-432f-ac95-074064234519 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:21.855745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1698c090-f1d5-4ae4-a96b-18719efceb88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.241679 Oct 28 09:49:21.856323 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 291/597] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:21 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:21.860601 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f68878f6-9c69-432f-ac95-074064234519 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:21.861072 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f68878f6-9c69-432f-ac95-074064234519 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:21.861500 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f68878f6-9c69-432f-ac95-074064234519 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:21.862185 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f68878f6-9c69-432f-ac95-074064234519 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:21.862671 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f68878f6-9c69-432f-ac95-074064234519 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:21.863166 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f68878f6-9c69-432f-ac95-074064234519 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:21.893565 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-f68878f6-9c69-432f-ac95-074064234519 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044614 Oct 28 09:49:21.894107 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 307/598] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:49:21 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:22.878010 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-08c94524-577a-4ef9-93a1-c11d7cc94635 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:22.878660 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-08c94524-577a-4ef9-93a1-c11d7cc94635 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:22.881144 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-ff0bd9cb-be89-48cf-9521-d700366b022e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:22.881757 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-ff0bd9cb-be89-48cf-9521-d700366b022e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:22.885524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-08c94524-577a-4ef9-93a1-c11d7cc94635 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:22.886006 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-08c94524-577a-4ef9-93a1-c11d7cc94635 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:22.886463 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-08c94524-577a-4ef9-93a1-c11d7cc94635 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:22.887195 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-08c94524-577a-4ef9-93a1-c11d7cc94635 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:22.887654 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-08c94524-577a-4ef9-93a1-c11d7cc94635 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:22.888054 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-08c94524-577a-4ef9-93a1-c11d7cc94635 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:22.888308 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ff0bd9cb-be89-48cf-9521-d700366b022e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:22.888799 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ff0bd9cb-be89-48cf-9521-d700366b022e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:22.889260 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ff0bd9cb-be89-48cf-9521-d700366b022e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:22.889935 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ff0bd9cb-be89-48cf-9521-d700366b022e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:22.890413 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ff0bd9cb-be89-48cf-9521-d700366b022e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:22.891744 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ff0bd9cb-be89-48cf-9521-d700366b022e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:23.094193 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-08c94524-577a-4ef9-93a1-c11d7cc94635 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:23.099370 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-ff0bd9cb-be89-48cf-9521-d700366b022e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:23.103247 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-08c94524-577a-4ef9-93a1-c11d7cc94635 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.229742 Oct 28 09:49:23.103831 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 292/599] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:22 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:23.119753 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-ff0bd9cb-be89-48cf-9521-d700366b022e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.231511 Oct 28 09:49:23.120122 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3376167a-2781-4885-bc00-f96f4360413f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:23.120542 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 308/600] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:22 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:23.120647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3376167a-2781-4885-bc00-f96f4360413f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:23.120647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3376167a-2781-4885-bc00-f96f4360413f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:23.120647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3376167a-2781-4885-bc00-f96f4360413f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:23.120647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3376167a-2781-4885-bc00-f96f4360413f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:23.121297 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3376167a-2781-4885-bc00-f96f4360413f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:23.121297 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3376167a-2781-4885-bc00-f96f4360413f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:23.121297 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3376167a-2781-4885-bc00-f96f4360413f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:23.156072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3376167a-2781-4885-bc00-f96f4360413f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051183 Oct 28 09:49:23.156647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 293/601] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:49:23 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 09:49:23.227501 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-89d77f78-ef47-47ee-84e0-75d38502e288 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:23.228016 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-89d77f78-ef47-47ee-84e0-75d38502e288 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:23.234282 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-89d77f78-ef47-47ee-84e0-75d38502e288 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:23.234820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-89d77f78-ef47-47ee-84e0-75d38502e288 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:23.235250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-89d77f78-ef47-47ee-84e0-75d38502e288 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:23.236630 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-89d77f78-ef47-47ee-84e0-75d38502e288 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:23.237078 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-89d77f78-ef47-47ee-84e0-75d38502e288 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:23.237524 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-89d77f78-ef47-47ee-84e0-75d38502e288 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:23.265159 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-89d77f78-ef47-47ee-84e0-75d38502e288 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Going to try to terminate instance {{(pid=127029) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:49:23.354650 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-89d77f78-ef47-47ee-84e0-75d38502e288 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 204 len: 0 microversion: 2.1 time: 0.130624 Oct 28 09:49:23.355521 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 309/602] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:49:23 2022] DELETE /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 0 bytes in 133 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 28 09:49:23.365696 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-575354b0-d52a-450f-94e9-09ccbe74c566 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:23.366436 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-575354b0-d52a-450f-94e9-09ccbe74c566 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:23.373201 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-575354b0-d52a-450f-94e9-09ccbe74c566 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:23.373692 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-575354b0-d52a-450f-94e9-09ccbe74c566 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:23.374115 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-575354b0-d52a-450f-94e9-09ccbe74c566 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:23.374839 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-575354b0-d52a-450f-94e9-09ccbe74c566 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:23.375290 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-575354b0-d52a-450f-94e9-09ccbe74c566 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:23.375828 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-575354b0-d52a-450f-94e9-09ccbe74c566 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:23.519519 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-575354b0-d52a-450f-94e9-09ccbe74c566 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:23.529478 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-575354b0-d52a-450f-94e9-09ccbe74c566 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1682 microversion: 2.1 time: 0.167963 Oct 28 09:49:23.530055 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 294/603] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:23 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1682 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:24.133801 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-32f4fe8e-362b-4515-a0f9-23027b57f754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:24.134822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-32f4fe8e-362b-4515-a0f9-23027b57f754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:24.135189 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-33f3a1bf-b612-4267-9c51-066ba7f515fb tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:24.135817 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-33f3a1bf-b612-4267-9c51-066ba7f515fb tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:24.141510 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-32f4fe8e-362b-4515-a0f9-23027b57f754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:24.141990 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-32f4fe8e-362b-4515-a0f9-23027b57f754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:24.142945 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-32f4fe8e-362b-4515-a0f9-23027b57f754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:24.143467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-33f3a1bf-b612-4267-9c51-066ba7f515fb tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:24.143638 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-32f4fe8e-362b-4515-a0f9-23027b57f754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:24.143950 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-33f3a1bf-b612-4267-9c51-066ba7f515fb tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:24.144064 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-32f4fe8e-362b-4515-a0f9-23027b57f754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:24.144377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-33f3a1bf-b612-4267-9c51-066ba7f515fb tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:24.144475 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-32f4fe8e-362b-4515-a0f9-23027b57f754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:24.145067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-33f3a1bf-b612-4267-9c51-066ba7f515fb tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:24.145511 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-33f3a1bf-b612-4267-9c51-066ba7f515fb tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:24.145926 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-33f3a1bf-b612-4267-9c51-066ba7f515fb tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:24.345084 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-32f4fe8e-362b-4515-a0f9-23027b57f754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:24.351623 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-33f3a1bf-b612-4267-9c51-066ba7f515fb tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:24.354049 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-32f4fe8e-362b-4515-a0f9-23027b57f754 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.224703 Oct 28 09:49:24.354776 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 310/604] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:24 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:24.359505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-33f3a1bf-b612-4267-9c51-066ba7f515fb tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.228100 Oct 28 09:49:24.360627 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 295/605] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:24 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:24.553923 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b30d1932-7ed3-4ca5-bf9c-1e2891b318c5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:24.554461 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b30d1932-7ed3-4ca5-bf9c-1e2891b318c5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:24.560603 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b30d1932-7ed3-4ca5-bf9c-1e2891b318c5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:24.561082 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b30d1932-7ed3-4ca5-bf9c-1e2891b318c5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:24.561510 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b30d1932-7ed3-4ca5-bf9c-1e2891b318c5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:24.562243 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b30d1932-7ed3-4ca5-bf9c-1e2891b318c5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:24.562730 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b30d1932-7ed3-4ca5-bf9c-1e2891b318c5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:24.563145 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b30d1932-7ed3-4ca5-bf9c-1e2891b318c5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:24.707985 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b30d1932-7ed3-4ca5-bf9c-1e2891b318c5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:24.716213 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b30d1932-7ed3-4ca5-bf9c-1e2891b318c5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1682 microversion: 2.1 time: 0.166176 Oct 28 09:49:24.716797 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 311/606] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:24 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1682 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:25.378456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8150fda4-7158-4944-8c64-a62d3c0a96bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:25.379072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8150fda4-7158-4944-8c64-a62d3c0a96bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:25.384553 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-ed3b2ee6-07ca-4e10-8ce8-efe68d40ec34 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:25.385172 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-ed3b2ee6-07ca-4e10-8ce8-efe68d40ec34 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:25.385984 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8150fda4-7158-4944-8c64-a62d3c0a96bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:25.386539 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8150fda4-7158-4944-8c64-a62d3c0a96bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:25.386979 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8150fda4-7158-4944-8c64-a62d3c0a96bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:25.387681 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8150fda4-7158-4944-8c64-a62d3c0a96bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:25.388131 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8150fda4-7158-4944-8c64-a62d3c0a96bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:25.388548 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8150fda4-7158-4944-8c64-a62d3c0a96bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:25.391399 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ed3b2ee6-07ca-4e10-8ce8-efe68d40ec34 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:25.391876 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ed3b2ee6-07ca-4e10-8ce8-efe68d40ec34 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:25.392906 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ed3b2ee6-07ca-4e10-8ce8-efe68d40ec34 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:25.393596 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ed3b2ee6-07ca-4e10-8ce8-efe68d40ec34 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:25.394048 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ed3b2ee6-07ca-4e10-8ce8-efe68d40ec34 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:25.394495 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ed3b2ee6-07ca-4e10-8ce8-efe68d40ec34 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:25.592753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8150fda4-7158-4944-8c64-a62d3c0a96bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:25.598490 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-ed3b2ee6-07ca-4e10-8ce8-efe68d40ec34 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:25.606771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-ed3b2ee6-07ca-4e10-8ce8-efe68d40ec34 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.226426 Oct 28 09:49:25.607402 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 312/607] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:25 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:25.610051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8150fda4-7158-4944-8c64-a62d3c0a96bb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.235511 Oct 28 09:49:25.612311 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 296/608] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:25 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:25.740502 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-01d3d1d8-166e-4426-8ecc-1e14e34764f5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:25.741022 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-01d3d1d8-166e-4426-8ecc-1e14e34764f5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:25.747063 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-01d3d1d8-166e-4426-8ecc-1e14e34764f5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:25.747543 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-01d3d1d8-166e-4426-8ecc-1e14e34764f5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:25.747973 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-01d3d1d8-166e-4426-8ecc-1e14e34764f5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:25.748657 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-01d3d1d8-166e-4426-8ecc-1e14e34764f5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:25.749105 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-01d3d1d8-166e-4426-8ecc-1e14e34764f5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:25.749527 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-01d3d1d8-166e-4426-8ecc-1e14e34764f5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:25.882940 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-01d3d1d8-166e-4426-8ecc-1e14e34764f5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:25.891533 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-01d3d1d8-166e-4426-8ecc-1e14e34764f5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1682 microversion: 2.1 time: 0.155368 Oct 28 09:49:25.892239 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 313/609] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:25 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1682 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:26.633660 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c959b39b-2f02-4f52-ba98-d82cb8646b6e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:26.634354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c959b39b-2f02-4f52-ba98-d82cb8646b6e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:26.636878 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-332692a1-8c05-4990-84e1-000fd34023b1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:26.637485 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-332692a1-8c05-4990-84e1-000fd34023b1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:26.641594 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c959b39b-2f02-4f52-ba98-d82cb8646b6e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:26.642569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c959b39b-2f02-4f52-ba98-d82cb8646b6e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:26.643024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c959b39b-2f02-4f52-ba98-d82cb8646b6e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:26.643729 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c959b39b-2f02-4f52-ba98-d82cb8646b6e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:26.644175 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c959b39b-2f02-4f52-ba98-d82cb8646b6e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:26.644745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c959b39b-2f02-4f52-ba98-d82cb8646b6e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:26.644883 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-332692a1-8c05-4990-84e1-000fd34023b1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:26.645227 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-332692a1-8c05-4990-84e1-000fd34023b1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:26.645657 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-332692a1-8c05-4990-84e1-000fd34023b1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:26.646347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-332692a1-8c05-4990-84e1-000fd34023b1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:26.646824 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-332692a1-8c05-4990-84e1-000fd34023b1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:26.647234 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-332692a1-8c05-4990-84e1-000fd34023b1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:26.848085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c959b39b-2f02-4f52-ba98-d82cb8646b6e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:26.851408 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-332692a1-8c05-4990-84e1-000fd34023b1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:26.856646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c959b39b-2f02-4f52-ba98-d82cb8646b6e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.227301 Oct 28 09:49:26.857229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 297/610] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:26 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:26.860196 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-332692a1-8c05-4990-84e1-000fd34023b1 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.226973 Oct 28 09:49:26.860910 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 314/611] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:26 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:26.915482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d466e023-048d-4b6b-a23e-49c7a2f2addb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:26.916495 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d466e023-048d-4b6b-a23e-49c7a2f2addb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:26.923883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d466e023-048d-4b6b-a23e-49c7a2f2addb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:26.924378 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d466e023-048d-4b6b-a23e-49c7a2f2addb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:26.924813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d466e023-048d-4b6b-a23e-49c7a2f2addb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:26.925523 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d466e023-048d-4b6b-a23e-49c7a2f2addb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:26.925973 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d466e023-048d-4b6b-a23e-49c7a2f2addb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:26.926502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d466e023-048d-4b6b-a23e-49c7a2f2addb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:27.101401 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d466e023-048d-4b6b-a23e-49c7a2f2addb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:27.112838 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d466e023-048d-4b6b-a23e-49c7a2f2addb tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1682 microversion: 2.1 time: 0.201519 Oct 28 09:49:27.113446 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 298/612] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:26 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1682 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:27.880947 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-1d6c16d6-3c6e-4e1f-a458-b3b930b70de9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:27.881462 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-1d6c16d6-3c6e-4e1f-a458-b3b930b70de9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:27.884471 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c683483c-be0d-45c4-b3a5-248e43c543b3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:27.885604 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c683483c-be0d-45c4-b3a5-248e43c543b3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:27.889103 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1d6c16d6-3c6e-4e1f-a458-b3b930b70de9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:27.889595 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1d6c16d6-3c6e-4e1f-a458-b3b930b70de9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:27.890558 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1d6c16d6-3c6e-4e1f-a458-b3b930b70de9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:27.890763 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1d6c16d6-3c6e-4e1f-a458-b3b930b70de9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:27.891206 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1d6c16d6-3c6e-4e1f-a458-b3b930b70de9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:27.891626 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1d6c16d6-3c6e-4e1f-a458-b3b930b70de9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:27.895055 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c683483c-be0d-45c4-b3a5-248e43c543b3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:27.896686 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c683483c-be0d-45c4-b3a5-248e43c543b3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:27.896686 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c683483c-be0d-45c4-b3a5-248e43c543b3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:27.897894 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c683483c-be0d-45c4-b3a5-248e43c543b3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:27.898726 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c683483c-be0d-45c4-b3a5-248e43c543b3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:27.899471 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c683483c-be0d-45c4-b3a5-248e43c543b3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:28.118692 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-1d6c16d6-3c6e-4e1f-a458-b3b930b70de9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:28.130845 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-1d6c16d6-3c6e-4e1f-a458-b3b930b70de9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1677 microversion: 2.1 time: 0.254267 Oct 28 09:49:28.131778 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 315/613] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:27 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1677 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:28.134432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c683483c-be0d-45c4-b3a5-248e43c543b3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:28.147228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c683483c-be0d-45c4-b3a5-248e43c543b3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.266484 Oct 28 09:49:28.148686 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 299/614] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:27 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:28.154551 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-0356aed1-374c-420b-bc09-4a3e121b4689 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:28.156034 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-0356aed1-374c-420b-bc09-4a3e121b4689 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:28.167119 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0356aed1-374c-420b-bc09-4a3e121b4689 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:28.168425 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0356aed1-374c-420b-bc09-4a3e121b4689 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:28.169470 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0356aed1-374c-420b-bc09-4a3e121b4689 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:28.171289 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0356aed1-374c-420b-bc09-4a3e121b4689 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:28.172191 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0356aed1-374c-420b-bc09-4a3e121b4689 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:28.172976 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0356aed1-374c-420b-bc09-4a3e121b4689 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:28.442246 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0356aed1-374c-420b-bc09-4a3e121b4689 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:28.451395 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-0356aed1-374c-420b-bc09-4a3e121b4689 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1682 microversion: 2.1 time: 0.310179 Oct 28 09:49:28.452000 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 316/615] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:28 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1682 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:29.159686 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a636b889-64aa-4803-ac0f-6e28989e81b1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:29.163733 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a636b889-64aa-4803-ac0f-6e28989e81b1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:29.173302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a636b889-64aa-4803-ac0f-6e28989e81b1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:29.174034 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a636b889-64aa-4803-ac0f-6e28989e81b1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:29.174826 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a636b889-64aa-4803-ac0f-6e28989e81b1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:29.175928 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a636b889-64aa-4803-ac0f-6e28989e81b1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:29.177001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a636b889-64aa-4803-ac0f-6e28989e81b1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:29.177572 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a636b889-64aa-4803-ac0f-6e28989e81b1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:29.182553 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:29.196442 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e76c53ee-8695-4a7c-a47d-53a6561519e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:29.198694 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e76c53ee-8695-4a7c-a47d-53a6561519e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:29.211757 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e76c53ee-8695-4a7c-a47d-53a6561519e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:29.213108 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e76c53ee-8695-4a7c-a47d-53a6561519e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:29.214102 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e76c53ee-8695-4a7c-a47d-53a6561519e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:29.215861 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e76c53ee-8695-4a7c-a47d-53a6561519e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:29.234765 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e76c53ee-8695-4a7c-a47d-53a6561519e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:29.240381 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e76c53ee-8695-4a7c-a47d-53a6561519e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:29.456146 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a636b889-64aa-4803-ac0f-6e28989e81b1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:29.487534 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a636b889-64aa-4803-ac0f-6e28989e81b1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1654 microversion: 2.1 time: 0.333088 Oct 28 09:49:29.503117 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e76c53ee-8695-4a7c-a47d-53a6561519e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:29.503766 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 300/616] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:29 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1654 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:29.520885 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e76c53ee-8695-4a7c-a47d-53a6561519e7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.333620 Oct 28 09:49:29.534347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-0941adb3-6595-4be7-85d0-7c1f3f28bcdc req-8981d8cd-8476-4e8f-8c8d-db591ccb87c5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "82790f9d-85c2-4205-b9ff-290df00c2edf", "name": "network-vif-deleted", "tag": "392bf558-627c-4e33-82ed-38a82396d7b0"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:29.535564 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 317/617] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:29 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:29.540592 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:29.541218 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:29.552252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:29.552871 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:29.553318 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:29.554115 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:29.554636 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:29.555207 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:29.555415 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-0941adb3-6595-4be7-85d0-7c1f3f28bcdc req-8981d8cd-8476-4e8f-8c8d-db591ccb87c5 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:29.556166 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-0941adb3-6595-4be7-85d0-7c1f3f28bcdc req-8981d8cd-8476-4e8f-8c8d-db591ccb87c5 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:29.556823 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-0941adb3-6595-4be7-85d0-7c1f3f28bcdc req-8981d8cd-8476-4e8f-8c8d-db591ccb87c5 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:29.591653 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:29.592146 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:29.592585 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:29.593057 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.objects.instance [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lazy-loading 'fault' on Instance uuid 82790f9d-85c2-4205-b9ff-290df00c2edf {{(pid=127029) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:49:29.609241 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-0941adb3-6595-4be7-85d0-7c1f3f28bcdc req-8981d8cd-8476-4e8f-8c8d-db591ccb87c5 service nova] Creating event network-vif-deleted:392bf558-627c-4e33-82ed-38a82396d7b0 for instance 82790f9d-85c2-4205-b9ff-290df00c2edf on hv-cinder-zed-3 Oct 28 09:49:29.614684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-0941adb3-6595-4be7-85d0-7c1f3f28bcdc req-8981d8cd-8476-4e8f-8c8d-db591ccb87c5 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.109302 Oct 28 09:49:29.616001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 301/618] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:49:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:29.621834 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-51662b7a-6043-4c50-a63b-b8eaf4284e57 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:29.636417 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-51662b7a-6043-4c50-a63b-b8eaf4284e57 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:29.639156 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-51662b7a-6043-4c50-a63b-b8eaf4284e57 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:29.639621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-51662b7a-6043-4c50-a63b-b8eaf4284e57 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:29.651200 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:29.660268 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b8069dd4-b744-4022-830c-131e3fd7ebe5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 200 len: 1331 microversion: 2.1 time: 0.123538 Oct 28 09:49:29.660863 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 318/619] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:29 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 1331 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:29.673220 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-51662b7a-6043-4c50-a63b-b8eaf4284e57 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Going to try to start instance {{(pid=127030) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 28 09:49:29.724402 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-51662b7a-6043-4c50-a63b-b8eaf4284e57 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "POST /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/action" status: 202 len: 0 microversion: 2.1 time: 0.106609 Oct 28 09:49:29.724873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 302/620] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:49:29 2022] POST /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/action => generated 0 bytes in 108 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:49:29.737911 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e6874fc0-5def-4c13-9ed3-43c65d4bd8b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:29.738747 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e6874fc0-5def-4c13-9ed3-43c65d4bd8b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:29.749826 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e6874fc0-5def-4c13-9ed3-43c65d4bd8b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:29.751553 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e6874fc0-5def-4c13-9ed3-43c65d4bd8b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:29.752945 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e6874fc0-5def-4c13-9ed3-43c65d4bd8b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:29.754725 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e6874fc0-5def-4c13-9ed3-43c65d4bd8b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:29.755510 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e6874fc0-5def-4c13-9ed3-43c65d4bd8b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:29.756029 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e6874fc0-5def-4c13-9ed3-43c65d4bd8b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:29.906302 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e6874fc0-5def-4c13-9ed3-43c65d4bd8b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:29.910091 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:29.915382 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e6874fc0-5def-4c13-9ed3-43c65d4bd8b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1663 microversion: 2.1 time: 0.184572 Oct 28 09:49:29.915965 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 319/621] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:29 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1663 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:30.546524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f26979e0-0334-4d07-b0f2-8fd62d480fb8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:30.547090 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f26979e0-0334-4d07-b0f2-8fd62d480fb8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:30.556618 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d645be60-2b8f-4716-b75c-6a0c5282caaa tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:30.557368 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d645be60-2b8f-4716-b75c-6a0c5282caaa tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:30.561859 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f26979e0-0334-4d07-b0f2-8fd62d480fb8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:30.562291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f26979e0-0334-4d07-b0f2-8fd62d480fb8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:30.562955 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f26979e0-0334-4d07-b0f2-8fd62d480fb8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:30.563833 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f26979e0-0334-4d07-b0f2-8fd62d480fb8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:30.564326 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f26979e0-0334-4d07-b0f2-8fd62d480fb8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:30.564749 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f26979e0-0334-4d07-b0f2-8fd62d480fb8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:30.565669 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d645be60-2b8f-4716-b75c-6a0c5282caaa tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:30.566223 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d645be60-2b8f-4716-b75c-6a0c5282caaa tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:30.567569 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d645be60-2b8f-4716-b75c-6a0c5282caaa tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:30.568512 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d645be60-2b8f-4716-b75c-6a0c5282caaa tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:30.570792 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d645be60-2b8f-4716-b75c-6a0c5282caaa tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:30.571526 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d645be60-2b8f-4716-b75c-6a0c5282caaa tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:30.658705 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-f26979e0-0334-4d07-b0f2-8fd62d480fb8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lazy-loading 'services' on Instance uuid 85327eca-6324-41a4-8b03-f2d5226cf2f5 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:49:30.739540 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f26979e0-0334-4d07-b0f2-8fd62d480fb8 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "DELETE /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5/os-volume_attachments/f61bb8b5-c2d8-4319-b39b-25de29ecd6bc" status: 202 len: 0 microversion: 2.1 time: 0.197813 Oct 28 09:49:30.740158 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 303/622] 10.222.0.25 () {58 vars in 1416 bytes} [Fri Oct 28 09:49:30 2022] DELETE /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5/os-volume_attachments/f61bb8b5-c2d8-4319-b39b-25de29ecd6bc => generated 0 bytes in 199 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:49:30.748580 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-59a1c46c-363c-4e64-aa05-9edbdd5f1d3c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:30.749286 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-59a1c46c-363c-4e64-aa05-9edbdd5f1d3c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:30.756143 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a1c46c-363c-4e64-aa05-9edbdd5f1d3c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:30.756715 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a1c46c-363c-4e64-aa05-9edbdd5f1d3c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:30.757207 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a1c46c-363c-4e64-aa05-9edbdd5f1d3c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:30.758068 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a1c46c-363c-4e64-aa05-9edbdd5f1d3c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:30.758555 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a1c46c-363c-4e64-aa05-9edbdd5f1d3c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:30.759167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a1c46c-363c-4e64-aa05-9edbdd5f1d3c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:30.774809 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-59a1c46c-363c-4e64-aa05-9edbdd5f1d3c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-59a1c46c-363c-4e64-aa05-9edbdd5f1d3c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf HTTP/1.1 Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:49:30.777051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:49:30.778751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:49:30.778751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:49:30.778751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:49:30.778751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:49:30.778751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:49:30.778751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:49:30.778751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:49:30.778751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:49:30.778751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:49:30.778751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:49:30.778751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:49:30.778751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '82790f9d-85c2-4205-b9ff-290df00c2edf'} Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:49:30.779528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:49:30.781016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:30.784304 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-59a1c46c-363c-4e64-aa05-9edbdd5f1d3c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 404 to user: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:49:30.785389 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-59a1c46c-363c-4e64-aa05-9edbdd5f1d3c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 404 len: 111 microversion: 2.1 time: 0.044085 Oct 28 09:49:30.786561 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 304/623] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:30 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:49:30.795399 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2b425fe2-571f-40da-b159-86bb42ee712b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '.version_select at 0x7f271a395d30>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:30.826417 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2b425fe2-571f-40da-b159-86bb42ee712b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-428074548" status: 202 len: 0 microversion: 2.1 time: 0.034422 Oct 28 09:49:30.827009 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 305/624] 10.222.0.25 () {58 vars in 1218 bytes} [Fri Oct 28 09:49:30 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-428074548 => generated 0 bytes in 36 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 09:49:30.861742 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d645be60-2b8f-4716-b75c-6a0c5282caaa tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:30.871345 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d645be60-2b8f-4716-b75c-6a0c5282caaa tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.321209 Oct 28 09:49:30.872177 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 320/625] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:30 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:30.939068 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3746f154-c23b-444c-9b7a-3d0058794334 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:30.939680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3746f154-c23b-444c-9b7a-3d0058794334 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:30.946813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3746f154-c23b-444c-9b7a-3d0058794334 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:30.947301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3746f154-c23b-444c-9b7a-3d0058794334 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:30.947733 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3746f154-c23b-444c-9b7a-3d0058794334 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:30.948510 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3746f154-c23b-444c-9b7a-3d0058794334 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:30.949005 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3746f154-c23b-444c-9b7a-3d0058794334 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:30.949430 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3746f154-c23b-444c-9b7a-3d0058794334 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:31.219712 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3746f154-c23b-444c-9b7a-3d0058794334 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:31.227970 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3746f154-c23b-444c-9b7a-3d0058794334 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1663 microversion: 2.1 time: 0.292828 Oct 28 09:49:31.228533 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 306/626] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:30 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1663 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:31.559991 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-dd0ea5e3-7731-4bd6-9d31-8b2163ffebc2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Action: 'create', calling method: .version_select at 0x7f271a2f30d0>, body: {"keypair": {"name": "tempest-keypair-2064819975", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNXvr+CoAJQHK4lHCpxKp9Z6B9WtOAOe5KoxV9JW5ZAZxfVbvrvceD7kWZH/m1vxXo7Mz0wSj9BLGXldr+aEsCQUNpSSON+TFrAFm1uoZWpAfIIPwsh3v8BD0xlLwgTETA=="}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:31.579415 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-dd0ea5e3-7731-4bd6-9d31-8b2163ffebc2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for project 4f351d016ace4e369be852bb7aa916fe. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:49:31.584106 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-dd0ea5e3-7731-4bd6-9d31-8b2163ffebc2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for user e65d298cca0f478eba502edf5bd13954 and project 4f351d016ace4e369be852bb7aa916fe. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:49:31.607781 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-dd0ea5e3-7731-4bd6-9d31-8b2163ffebc2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.052415 Oct 28 09:49:31.608380 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 321/627] 10.222.0.25 () {60 vars in 1159 bytes} [Fri Oct 28 09:49:31 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:31.898022 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ea8157a5-8cf9-4766-9dc8-e75805694f1c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:31.898957 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ea8157a5-8cf9-4766-9dc8-e75805694f1c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:31.905857 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ea8157a5-8cf9-4766-9dc8-e75805694f1c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:31.906316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ea8157a5-8cf9-4766-9dc8-e75805694f1c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:31.906949 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ea8157a5-8cf9-4766-9dc8-e75805694f1c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:31.907988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ea8157a5-8cf9-4766-9dc8-e75805694f1c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:31.908936 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ea8157a5-8cf9-4766-9dc8-e75805694f1c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:31.909545 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ea8157a5-8cf9-4766-9dc8-e75805694f1c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:32.112809 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ea8157a5-8cf9-4766-9dc8-e75805694f1c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:32.121434 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ea8157a5-8cf9-4766-9dc8-e75805694f1c tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.228012 Oct 28 09:49:32.122006 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 307/628] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:31 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:32.253188 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-ef8b7268-5ff1-4442-a54b-dcf827bda2c8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:32.253849 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-ef8b7268-5ff1-4442-a54b-dcf827bda2c8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:32.260920 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ef8b7268-5ff1-4442-a54b-dcf827bda2c8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:32.261408 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ef8b7268-5ff1-4442-a54b-dcf827bda2c8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:32.261842 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ef8b7268-5ff1-4442-a54b-dcf827bda2c8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:32.262752 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ef8b7268-5ff1-4442-a54b-dcf827bda2c8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:32.263452 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ef8b7268-5ff1-4442-a54b-dcf827bda2c8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:32.263936 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ef8b7268-5ff1-4442-a54b-dcf827bda2c8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:32.566215 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-ef8b7268-5ff1-4442-a54b-dcf827bda2c8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:32.575361 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-ef8b7268-5ff1-4442-a54b-dcf827bda2c8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1667 microversion: 2.1 time: 0.326593 Oct 28 09:49:32.576213 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 322/629] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:32 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1667 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:32.942475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2c30f11e-5476-4fd9-9130-a05bd579a97b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:32.943099 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2c30f11e-5476-4fd9-9130-a05bd579a97b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:32.949707 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2c30f11e-5476-4fd9-9130-a05bd579a97b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:32.950184 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2c30f11e-5476-4fd9-9130-a05bd579a97b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:32.950664 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2c30f11e-5476-4fd9-9130-a05bd579a97b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:32.951450 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2c30f11e-5476-4fd9-9130-a05bd579a97b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:32.951838 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2c30f11e-5476-4fd9-9130-a05bd579a97b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:32.952259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2c30f11e-5476-4fd9-9130-a05bd579a97b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:32.978676 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2c30f11e-5476-4fd9-9130-a05bd579a97b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Going to try to terminate instance {{(pid=127030) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:49:33.061313 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2c30f11e-5476-4fd9-9130-a05bd579a97b tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "DELETE /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 204 len: 0 microversion: 2.1 time: 0.122325 Oct 28 09:49:33.061970 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 308/630] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:49:32 2022] DELETE /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:49:33.070065 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-ef39b3b5-150f-4b8b-ad0d-7d8024e0abe6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:33.070750 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-ef39b3b5-150f-4b8b-ad0d-7d8024e0abe6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:33.077197 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ef39b3b5-150f-4b8b-ad0d-7d8024e0abe6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:33.077608 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ef39b3b5-150f-4b8b-ad0d-7d8024e0abe6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:33.078073 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ef39b3b5-150f-4b8b-ad0d-7d8024e0abe6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:33.079109 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ef39b3b5-150f-4b8b-ad0d-7d8024e0abe6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:33.080189 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ef39b3b5-150f-4b8b-ad0d-7d8024e0abe6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:33.080633 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-ef39b3b5-150f-4b8b-ad0d-7d8024e0abe6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:33.147030 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ef7e3f68-1666-4168-ba13-3eb29aba0e9d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:33.147730 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ef7e3f68-1666-4168-ba13-3eb29aba0e9d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:33.154018 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef7e3f68-1666-4168-ba13-3eb29aba0e9d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:33.154535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef7e3f68-1666-4168-ba13-3eb29aba0e9d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:33.154967 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef7e3f68-1666-4168-ba13-3eb29aba0e9d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:33.155669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef7e3f68-1666-4168-ba13-3eb29aba0e9d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:33.156116 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef7e3f68-1666-4168-ba13-3eb29aba0e9d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:33.156533 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef7e3f68-1666-4168-ba13-3eb29aba0e9d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:33.339984 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-ef39b3b5-150f-4b8b-ad0d-7d8024e0abe6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:33.349746 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-ef39b3b5-150f-4b8b-ad0d-7d8024e0abe6 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1515 microversion: 2.1 time: 0.283453 Oct 28 09:49:33.350331 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 323/631] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:33 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1515 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:33.406025 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ef7e3f68-1666-4168-ba13-3eb29aba0e9d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:33.414573 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ef7e3f68-1666-4168-ba13-3eb29aba0e9d tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.273201 Oct 28 09:49:33.415236 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 309/632] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:33 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:33.473220 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-928908247", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8cae7e07-cd54-4f98-96b4-a27c37dcb83a"}], "security_groups": [{"name": "tempest-securitygroup--1995095774"}], "key_name": "tempest-keypair-2064819975", "user_data": "}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:33.584931 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.network.neutron [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] validate_networks() for [('8cae7e07-cd54-4f98-96b4-a27c37dcb83a', None, None, None, None, None)] {{(pid=127029) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:49:33.966982 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Flavor limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:49:33.967405 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Image limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:49:33.967894 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Flavor pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:49:33.968280 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Image pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:49:33.968686 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:49:33.978307 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:33.978813 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:33.979245 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:33.979963 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:33.980410 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:33.980831 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:33.997121 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for project 4f351d016ace4e369be852bb7aa916fe. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:49:34.001313 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for user e65d298cca0f478eba502edf5bd13954 and project 4f351d016ace4e369be852bb7aa916fe. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:49:34.012927 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Going to run 1 instances... {{(pid=127029) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:49:34.015629 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127029) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:49:34.065481 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-11889dd9-7d1c-4173-9870-cd67a30530b4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.597123 Oct 28 09:49:34.066236 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 324/633] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:49:33 2022] POST /compute/v2.1/servers => generated 402 bytes in 599 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 09:49:34.077651 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-93ce30d7-883f-4052-85fc-ff0bc6f25c77 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:34.078241 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-93ce30d7-883f-4052-85fc-ff0bc6f25c77 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:34.139776 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-93ce30d7-883f-4052-85fc-ff0bc6f25c77 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:34.147636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-93ce30d7-883f-4052-85fc-ff0bc6f25c77 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1241 microversion: 2.1 time: 0.073797 Oct 28 09:49:34.148433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 310/634] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:34 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1241 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:34.374984 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-27e63939-ae80-4344-99ab-8b05f97b9832 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:34.375708 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-27e63939-ae80-4344-99ab-8b05f97b9832 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:34.383498 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-27e63939-ae80-4344-99ab-8b05f97b9832 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:34.384090 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-27e63939-ae80-4344-99ab-8b05f97b9832 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:34.384612 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-27e63939-ae80-4344-99ab-8b05f97b9832 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:34.385386 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-27e63939-ae80-4344-99ab-8b05f97b9832 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:34.385898 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-27e63939-ae80-4344-99ab-8b05f97b9832 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:34.386260 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-27e63939-ae80-4344-99ab-8b05f97b9832 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:34.439701 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5732064e-30c8-4566-9fb5-58eba1fb4c02 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:34.440542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5732064e-30c8-4566-9fb5-58eba1fb4c02 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:34.446734 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5732064e-30c8-4566-9fb5-58eba1fb4c02 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:34.447217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5732064e-30c8-4566-9fb5-58eba1fb4c02 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:34.447749 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5732064e-30c8-4566-9fb5-58eba1fb4c02 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:34.448482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5732064e-30c8-4566-9fb5-58eba1fb4c02 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:34.448932 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5732064e-30c8-4566-9fb5-58eba1fb4c02 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:34.450453 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5732064e-30c8-4566-9fb5-58eba1fb4c02 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:34.582922 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-27e63939-ae80-4344-99ab-8b05f97b9832 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:34.590805 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-27e63939-ae80-4344-99ab-8b05f97b9832 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1515 microversion: 2.1 time: 0.219626 Oct 28 09:49:34.591542 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 325/635] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:34 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1515 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:34.629514 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5732064e-30c8-4566-9fb5-58eba1fb4c02 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:34.637990 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5732064e-30c8-4566-9fb5-58eba1fb4c02 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.203113 Oct 28 09:49:34.638506 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 311/636] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:34 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:35.171432 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-4bef0b68-4712-468a-ab53-73a2cfaa1672 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:35.172035 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-4bef0b68-4712-468a-ab53-73a2cfaa1672 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:35.178686 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4bef0b68-4712-468a-ab53-73a2cfaa1672 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:35.179173 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4bef0b68-4712-468a-ab53-73a2cfaa1672 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:35.179601 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4bef0b68-4712-468a-ab53-73a2cfaa1672 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:35.180279 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4bef0b68-4712-468a-ab53-73a2cfaa1672 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:35.180728 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4bef0b68-4712-468a-ab53-73a2cfaa1672 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:35.181141 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4bef0b68-4712-468a-ab53-73a2cfaa1672 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:35.243219 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-4bef0b68-4712-468a-ab53-73a2cfaa1672 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:35.251015 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-4bef0b68-4712-468a-ab53-73a2cfaa1672 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1245 microversion: 2.1 time: 0.083749 Oct 28 09:49:35.252082 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 326/637] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:35 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1245 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:35.614634 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0d524b81-d93b-4b3c-a66f-e32d2d78225a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:35.615292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0d524b81-d93b-4b3c-a66f-e32d2d78225a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:35.622036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0d524b81-d93b-4b3c-a66f-e32d2d78225a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:35.622541 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0d524b81-d93b-4b3c-a66f-e32d2d78225a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:35.623014 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0d524b81-d93b-4b3c-a66f-e32d2d78225a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:35.623693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0d524b81-d93b-4b3c-a66f-e32d2d78225a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:35.624155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0d524b81-d93b-4b3c-a66f-e32d2d78225a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:35.624574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0d524b81-d93b-4b3c-a66f-e32d2d78225a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:35.661502 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d3048593-bcce-4096-ab49-fd0d3cd6ceec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:35.662112 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d3048593-bcce-4096-ab49-fd0d3cd6ceec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:35.668649 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3048593-bcce-4096-ab49-fd0d3cd6ceec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:35.669391 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3048593-bcce-4096-ab49-fd0d3cd6ceec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:35.669825 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3048593-bcce-4096-ab49-fd0d3cd6ceec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:35.670551 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3048593-bcce-4096-ab49-fd0d3cd6ceec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:35.671087 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3048593-bcce-4096-ab49-fd0d3cd6ceec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:35.671423 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d3048593-bcce-4096-ab49-fd0d3cd6ceec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:35.828327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0d524b81-d93b-4b3c-a66f-e32d2d78225a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:35.837671 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0d524b81-d93b-4b3c-a66f-e32d2d78225a tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1515 microversion: 2.1 time: 0.227003 Oct 28 09:49:35.838371 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 312/638] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:35 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1515 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:35.858236 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d3048593-bcce-4096-ab49-fd0d3cd6ceec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:35.866700 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d3048593-bcce-4096-ab49-fd0d3cd6ceec tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.209493 Oct 28 09:49:35.867269 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 327/639] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:35 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:36.274841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-eee68bd0-0cba-4590-9f15-09cefb037a76 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:36.275451 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-eee68bd0-0cba-4590-9f15-09cefb037a76 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:36.293564 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eee68bd0-0cba-4590-9f15-09cefb037a76 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:36.294477 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eee68bd0-0cba-4590-9f15-09cefb037a76 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:36.300957 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eee68bd0-0cba-4590-9f15-09cefb037a76 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:36.301714 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eee68bd0-0cba-4590-9f15-09cefb037a76 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:36.302167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eee68bd0-0cba-4590-9f15-09cefb037a76 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:36.303334 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eee68bd0-0cba-4590-9f15-09cefb037a76 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:36.381864 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-eee68bd0-0cba-4590-9f15-09cefb037a76 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:36.391487 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-eee68bd0-0cba-4590-9f15-09cefb037a76 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1293 microversion: 2.1 time: 0.120367 Oct 28 09:49:36.392185 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 313/640] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:36 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1293 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:36.724287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-92fca910-1ef6-4d22-ad7d-99911c09894f req-29dfcfef-d202-4021-bbf9-e41116429ff7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "85327eca-6324-41a4-8b03-f2d5226cf2f5", "name": "network-vif-deleted", "tag": "be115348-3000-4d12-9056-f68416196802"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:36.734762 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-92fca910-1ef6-4d22-ad7d-99911c09894f req-29dfcfef-d202-4021-bbf9-e41116429ff7 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:36.735230 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-92fca910-1ef6-4d22-ad7d-99911c09894f req-29dfcfef-d202-4021-bbf9-e41116429ff7 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:36.735680 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-92fca910-1ef6-4d22-ad7d-99911c09894f req-29dfcfef-d202-4021-bbf9-e41116429ff7 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:36.747288 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-92fca910-1ef6-4d22-ad7d-99911c09894f req-29dfcfef-d202-4021-bbf9-e41116429ff7 service nova] Creating event network-vif-deleted:be115348-3000-4d12-9056-f68416196802 for instance 85327eca-6324-41a4-8b03-f2d5226cf2f5 on hv-cinder-zed-3 Oct 28 09:49:36.752273 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-92fca910-1ef6-4d22-ad7d-99911c09894f req-29dfcfef-d202-4021-bbf9-e41116429ff7 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032561 Oct 28 09:49:36.752858 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 328/641] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:49:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:36.861292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8395c902-8123-41b9-8559-815fb568ad9c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:36.861900 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8395c902-8123-41b9-8559-815fb568ad9c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:36.868488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8395c902-8123-41b9-8559-815fb568ad9c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:36.868967 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8395c902-8123-41b9-8559-815fb568ad9c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:36.869398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8395c902-8123-41b9-8559-815fb568ad9c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:36.870093 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8395c902-8123-41b9-8559-815fb568ad9c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:36.870568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8395c902-8123-41b9-8559-815fb568ad9c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:36.871062 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8395c902-8123-41b9-8559-815fb568ad9c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:36.892614 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d8c3c16f-5b7f-4df3-9d48-b5d2637bee88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:36.893343 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d8c3c16f-5b7f-4df3-9d48-b5d2637bee88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:36.895313 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:INSERT=4 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:36.901567 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d8c3c16f-5b7f-4df3-9d48-b5d2637bee88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:36.902101 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d8c3c16f-5b7f-4df3-9d48-b5d2637bee88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:36.902570 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d8c3c16f-5b7f-4df3-9d48-b5d2637bee88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:36.903694 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d8c3c16f-5b7f-4df3-9d48-b5d2637bee88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:36.904172 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d8c3c16f-5b7f-4df3-9d48-b5d2637bee88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:36.904592 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d8c3c16f-5b7f-4df3-9d48-b5d2637bee88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:36.940291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8395c902-8123-41b9-8559-815fb568ad9c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:36.951643 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8395c902-8123-41b9-8559-815fb568ad9c tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1473 microversion: 2.1 time: 0.094146 Oct 28 09:49:36.952370 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 314/642] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:36 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1473 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:37.054803 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-d8c3c16f-5b7f-4df3-9d48-b5d2637bee88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:37.063783 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d8c3c16f-5b7f-4df3-9d48-b5d2637bee88 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1736 microversion: 2.1 time: 0.176034 Oct 28 09:49:37.064532 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 329/643] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:36 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1736 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:37.414642 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-64f76146-85fe-4c17-a038-b10bf9228a18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:37.415246 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-64f76146-85fe-4c17-a038-b10bf9228a18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:37.421539 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64f76146-85fe-4c17-a038-b10bf9228a18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:37.422013 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64f76146-85fe-4c17-a038-b10bf9228a18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:37.422461 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64f76146-85fe-4c17-a038-b10bf9228a18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:37.423145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64f76146-85fe-4c17-a038-b10bf9228a18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:37.424167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64f76146-85fe-4c17-a038-b10bf9228a18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:37.424583 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64f76146-85fe-4c17-a038-b10bf9228a18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:37.492810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-64f76146-85fe-4c17-a038-b10bf9228a18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:37.501271 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-64f76146-85fe-4c17-a038-b10bf9228a18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1293 microversion: 2.1 time: 0.090363 Oct 28 09:49:37.502143 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 315/644] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:37 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1293 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:37.976121 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:37.976752 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:37.983387 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:37.983872 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:37.984306 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:37.985025 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:37.985476 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:37.985944 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:38.018413 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:38.018914 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:38.019424 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:38.019917 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.objects.instance [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lazy-loading 'fault' on Instance uuid 85327eca-6324-41a4-8b03-f2d5226cf2f5 {{(pid=127029) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:49:38.061498 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:38.069693 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e13387fc-4775-40ff-ba76-190a615feab2 tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1309 microversion: 2.1 time: 0.097475 Oct 28 09:49:38.070280 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 330/645] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:37 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1309 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:38.088811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6b846aae-cdc1-4e03-b6f0-d31805f43d9e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:38.089413 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6b846aae-cdc1-4e03-b6f0-d31805f43d9e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:38.097018 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b846aae-cdc1-4e03-b6f0-d31805f43d9e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:38.097492 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b846aae-cdc1-4e03-b6f0-d31805f43d9e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:38.097919 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b846aae-cdc1-4e03-b6f0-d31805f43d9e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:38.098697 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b846aae-cdc1-4e03-b6f0-d31805f43d9e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:38.099139 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b846aae-cdc1-4e03-b6f0-d31805f43d9e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:38.099552 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b846aae-cdc1-4e03-b6f0-d31805f43d9e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:38.249490 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6b846aae-cdc1-4e03-b6f0-d31805f43d9e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:38.253100 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:38.260026 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6b846aae-cdc1-4e03-b6f0-d31805f43d9e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1716 microversion: 2.1 time: 0.175269 Oct 28 09:49:38.260738 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 316/646] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:38 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1716 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:38.388573 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-9ebad72b-0af0-4af1-af29-3c4c5f91e22f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:38.389643 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-9ebad72b-0af0-4af1-af29-3c4c5f91e22f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:38.400665 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9ebad72b-0af0-4af1-af29-3c4c5f91e22f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:38.401138 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9ebad72b-0af0-4af1-af29-3c4c5f91e22f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:38.401581 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9ebad72b-0af0-4af1-af29-3c4c5f91e22f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:38.402298 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9ebad72b-0af0-4af1-af29-3c4c5f91e22f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:38.402807 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9ebad72b-0af0-4af1-af29-3c4c5f91e22f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:38.403247 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9ebad72b-0af0-4af1-af29-3c4c5f91e22f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:38.496738 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.wsgi [None req-9ebad72b-0af0-4af1-af29-3c4c5f91e22f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] HTTP exception thrown: Cannot 'detach_volume' instance 4891b63b-eb08-448d-a640-f9579c8140ff while it is in vm_state rescued Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-9ebad72b-0af0-4af1-af29-3c4c5f91e22f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Request method failure captured: Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: request: DELETE /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-volume_attachments/855268e6-c6f2-4ab9-a6e6-79c92b1e0b32 HTTP/1.1 Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept: application/json Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept-Encoding: identity Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Connection: close Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Length: 0 Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Type: application/json Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Host: 10.222.0.25 Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Openstack-System-Scope: None Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: User-Agent: python-urllib3/1.26.12 Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Auth-Token: gAAAAABjW6Sjq718s-6y9fBFjvzR8wDyF_4AlWYEmdP4icnuJ0s_5turdxYbjJNH08Q4u87xp47MpEwc_KXLuW4AwBVGzdBvRRVA1GVRVASenhMGF1IGhESh28OTGp6LsG6ZaKxtUJblyS6zcH0d96QlXz7dkrheK75c4DcQ-AxWdIWnFfDpBXE Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Id: None Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Name: None Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Identity-Status: Confirmed Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Is-Admin-Project: True Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Id: default Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Name: Default Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Id: 227ac6c4e642467ca3b9e0bf7181d5c9 Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Role: member,reader Oct 28 09:49:38.498443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Roles: member,reader Oct 28 09:49:38.500062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/227ac6c4e642467ca3b9e0bf7181d5c9"}]}] Oct 28 09:49:38.500062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:49:38.500062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Id: 227ac6c4e642467ca3b9e0bf7181d5c9 Oct 28 09:49:38.500062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:49:38.500062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User: tempest-ServerRescueNegativeTestJSON-1296287768-project Oct 28 09:49:38.500062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Id: default Oct 28 09:49:38.500062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Name: Default Oct 28 09:49:38.500062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Id: ed9a4c24cbfb4689b7f984219c7503a4 Oct 28 09:49:38.500062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1296287768-project Oct 28 09:49:38.500062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: method: > Oct 28 09:49:38.500062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: exception: Cannot 'detach_volume' instance 4891b63b-eb08-448d-a640-f9579c8140ff while it is in vm_state rescued Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: action_args: {'server_id': '4891b63b-eb08-448d-a640-f9579c8140ff', 'id': '855268e6-c6f2-4ab9-a6e6-79c92b1e0b32'} Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 4891b63b-eb08-448d-a640-f9579c8140ff in vm_state rescued. Cannot detach_volume while the instance is in this state. Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:49:38.505347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 4891b63b-eb08-448d-a640-f9579c8140ff while it is in vm_state rescued Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 4891b63b-eb08-448d-a640-f9579c8140ff while it is in vm_state rescued Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:49:38.506820 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-9ebad72b-0af0-4af1-af29-3c4c5f91e22f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Returning 409 to user: Cannot 'detach_volume' instance 4891b63b-eb08-448d-a640-f9579c8140ff while it is in vm_state rescued {{(pid=127029) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:49:38.507932 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-9ebad72b-0af0-4af1-af29-3c4c5f91e22f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "DELETE /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-volume_attachments/855268e6-c6f2-4ab9-a6e6-79c92b1e0b32" status: 409 len: 152 microversion: 2.1 time: 0.115612 Oct 28 09:49:38.507932 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 331/647] 10.222.0.25 () {58 vars in 1416 bytes} [Fri Oct 28 09:49:38 2022] DELETE /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-volume_attachments/855268e6-c6f2-4ab9-a6e6-79c92b1e0b32 => generated 152 bytes in 119 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 28 09:49:38.515366 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-24934223-d5f1-4f0f-bb17-b9ecf5e1798a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:38.516128 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-24934223-d5f1-4f0f-bb17-b9ecf5e1798a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:38.525541 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24934223-d5f1-4f0f-bb17-b9ecf5e1798a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:38.525797 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3511a29d-bcf3-441c-b853-1116853e06cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:38.526184 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24934223-d5f1-4f0f-bb17-b9ecf5e1798a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:38.526467 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-3511a29d-bcf3-441c-b853-1116853e06cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:38.527220 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24934223-d5f1-4f0f-bb17-b9ecf5e1798a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:38.528014 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24934223-d5f1-4f0f-bb17-b9ecf5e1798a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:38.528540 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24934223-d5f1-4f0f-bb17-b9ecf5e1798a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:38.529034 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24934223-d5f1-4f0f-bb17-b9ecf5e1798a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:38.546704 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3511a29d-bcf3-441c-b853-1116853e06cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:38.547206 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3511a29d-bcf3-441c-b853-1116853e06cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:38.547650 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3511a29d-bcf3-441c-b853-1116853e06cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:38.548406 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3511a29d-bcf3-441c-b853-1116853e06cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:38.548885 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3511a29d-bcf3-441c-b853-1116853e06cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:38.549314 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3511a29d-bcf3-441c-b853-1116853e06cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:38.642706 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-24934223-d5f1-4f0f-bb17-b9ecf5e1798a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "POST /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/action" status: 202 len: 0 microversion: 2.1 time: 0.136045 Oct 28 09:49:38.643798 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 317/648] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:49:38 2022] POST /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/action => generated 0 bytes in 139 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 09:49:38.659026 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3511a29d-bcf3-441c-b853-1116853e06cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:38.661028 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ee19c1c0-5ebc-448e-95a0-99017de0ac66 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:38.661729 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ee19c1c0-5ebc-448e-95a0-99017de0ac66 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:38.677238 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-3511a29d-bcf3-441c-b853-1116853e06cd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1311 microversion: 2.1 time: 0.154912 Oct 28 09:49:38.677835 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 332/649] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:38 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1311 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:38.684095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ee19c1c0-5ebc-448e-95a0-99017de0ac66 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:38.684722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ee19c1c0-5ebc-448e-95a0-99017de0ac66 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:38.685133 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ee19c1c0-5ebc-448e-95a0-99017de0ac66 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:38.685837 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ee19c1c0-5ebc-448e-95a0-99017de0ac66 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:38.686280 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ee19c1c0-5ebc-448e-95a0-99017de0ac66 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:38.686726 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ee19c1c0-5ebc-448e-95a0-99017de0ac66 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:38.818203 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ee19c1c0-5ebc-448e-95a0-99017de0ac66 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:38.826646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ee19c1c0-5ebc-448e-95a0-99017de0ac66 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1724 microversion: 2.1 time: 0.169870 Oct 28 09:49:38.827245 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 318/650] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:38 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1724 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:39.093050 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:39.093847 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:39.102370 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:39.103105 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:39.103553 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:39.104254 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:39.104704 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:39.105118 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:39.141456 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:39.142543 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:39.142976 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:39.143420 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.objects.instance [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lazy-loading 'fault' on Instance uuid 85327eca-6324-41a4-8b03-f2d5226cf2f5 {{(pid=127029) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:49:39.194735 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec23c6593e7a446298d607294997664c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a20b51b10ce463da8f9ad280063bb09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:39.205273 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c15ca447-52f6-4d43-b343-46f9df05178d tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 200 len: 1309 microversion: 2.1 time: 0.115715 Oct 28 09:49:39.206150 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 333/651] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:39 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 1309 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:39.701987 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4688b776-bbdf-4a19-b7d5-858a3f6542fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:39.703333 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4688b776-bbdf-4a19-b7d5-858a3f6542fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:39.710827 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4688b776-bbdf-4a19-b7d5-858a3f6542fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:39.711314 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4688b776-bbdf-4a19-b7d5-858a3f6542fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:39.712349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4688b776-bbdf-4a19-b7d5-858a3f6542fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:39.713497 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4688b776-bbdf-4a19-b7d5-858a3f6542fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:39.714185 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4688b776-bbdf-4a19-b7d5-858a3f6542fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:39.714642 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4688b776-bbdf-4a19-b7d5-858a3f6542fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:39.850130 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6c72147d-5c83-48ba-bdba-b2151c0ef215 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:39.850809 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6c72147d-5c83-48ba-bdba-b2151c0ef215 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:39.857107 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6c72147d-5c83-48ba-bdba-b2151c0ef215 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:39.857859 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6c72147d-5c83-48ba-bdba-b2151c0ef215 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:39.858496 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6c72147d-5c83-48ba-bdba-b2151c0ef215 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:39.859365 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6c72147d-5c83-48ba-bdba-b2151c0ef215 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:39.860033 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6c72147d-5c83-48ba-bdba-b2151c0ef215 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:39.860407 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6c72147d-5c83-48ba-bdba-b2151c0ef215 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:40.297131 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-4688b776-bbdf-4a19-b7d5-858a3f6542fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:40.305514 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4688b776-bbdf-4a19-b7d5-858a3f6542fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.607994 Oct 28 09:49:40.306090 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 319/652] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:39 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:40.310951 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5598b55f-0f93-4a54-80b7-63ec3f69c2fb tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:40.311571 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5598b55f-0f93-4a54-80b7-63ec3f69c2fb tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] [instance: 85327eca-6324-41a4-8b03-f2d5226cf2f5] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:40.317692 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5598b55f-0f93-4a54-80b7-63ec3f69c2fb tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:40.318245 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5598b55f-0f93-4a54-80b7-63ec3f69c2fb tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:40.318725 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5598b55f-0f93-4a54-80b7-63ec3f69c2fb tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:40.319590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5598b55f-0f93-4a54-80b7-63ec3f69c2fb tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:40.320080 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5598b55f-0f93-4a54-80b7-63ec3f69c2fb tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:40.320509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5598b55f-0f93-4a54-80b7-63ec3f69c2fb tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:40.334726 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-5598b55f-0f93-4a54-80b7-63ec3f69c2fb tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] HTTP exception thrown: Instance 85327eca-6324-41a4-8b03-f2d5226cf2f5 could not be found. Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5598b55f-0f93-4a54-80b7-63ec3f69c2fb tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Request method failure captured: Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 HTTP/1.1 Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6UnJ0XS3DcbOAGlBvcokWOetHelVSNGAy0AA6ILn39s4_F70yEsLs6CjMI8eDJZzaXK_Vh-4HzBHeo6e1LWtOD2BwJHIifsVVqLySG6fHdXxgyxwCYeB2dy8v6CsCqwEus9FQmwu6YybRfmrg_mGOBhBtGsf1kxUvoqPIgxqYbOq4w Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 9a20b51b10ce463da8f9ad280063bb09 Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1142097989 Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:49:40.336722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:49:40.342535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/9a20b51b10ce463da8f9ad280063bb09"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/9a20b51b10ce463da8f9ad280063bb09"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_9a20b51b10ce463da8f9ad280063bb09"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/9a20b51b10ce463da8f9ad280063bb09"}]}] Oct 28 09:49:40.342535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-VolumesSnapshotTestJSON-1142097989 Oct 28 09:49:40.342535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 9a20b51b10ce463da8f9ad280063bb09 Oct 28 09:49:40.342535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1142097989 Oct 28 09:49:40.342535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-VolumesSnapshotTestJSON-1142097989-project Oct 28 09:49:40.342535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:49:40.342535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:49:40.342535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: ec23c6593e7a446298d607294997664c Oct 28 09:49:40.342535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-VolumesSnapshotTestJSON-1142097989-project Oct 28 09:49:40.342535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:49:40.342535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 85327eca-6324-41a4-8b03-f2d5226cf2f5 could not be found. Oct 28 09:49:40.342535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '85327eca-6324-41a4-8b03-f2d5226cf2f5'} Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 85327eca-6324-41a4-8b03-f2d5226cf2f5 could not be found. Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:49:40.343390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 85327eca-6324-41a4-8b03-f2d5226cf2f5 could not be found. Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 85327eca-6324-41a4-8b03-f2d5226cf2f5 could not be found. Oct 28 09:49:40.344647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:40.347103 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5598b55f-0f93-4a54-80b7-63ec3f69c2fb tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] Returning 404 to user: Instance 85327eca-6324-41a4-8b03-f2d5226cf2f5 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:49:40.347103 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5598b55f-0f93-4a54-80b7-63ec3f69c2fb tempest-VolumesSnapshotTestJSON-1142097989 tempest-VolumesSnapshotTestJSON-1142097989-project] 10.222.0.25 "GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5" status: 404 len: 111 microversion: 2.1 time: 0.031050 Oct 28 09:49:40.347103 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 320/653] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:40 2022] GET /compute/v2.1/servers/85327eca-6324-41a4-8b03-f2d5226cf2f5 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:49:40.459120 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6c72147d-5c83-48ba-bdba-b2151c0ef215 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:40.468263 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6c72147d-5c83-48ba-bdba-b2151c0ef215 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1724 microversion: 2.1 time: 0.622024 Oct 28 09:49:40.488099 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 334/654] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:39 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1724 bytes in 642 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:40.972660 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-a53809f3-be63-4a82-b99b-5b6388110b1c req-f014083c-8bf8-484c-9573-ff69b3796e81 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "576e3364-9c2e-4360-873a-7a9ded31e8e3", "tag": "b0640bfe-73f9-4cda-a3ed-720c31e1391d"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:40.982903 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-a53809f3-be63-4a82-b99b-5b6388110b1c req-f014083c-8bf8-484c-9573-ff69b3796e81 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:40.983359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-a53809f3-be63-4a82-b99b-5b6388110b1c req-f014083c-8bf8-484c-9573-ff69b3796e81 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:40.983801 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-a53809f3-be63-4a82-b99b-5b6388110b1c req-f014083c-8bf8-484c-9573-ff69b3796e81 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:40.995790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-a53809f3-be63-4a82-b99b-5b6388110b1c req-f014083c-8bf8-484c-9573-ff69b3796e81 service nova] Creating event network-changed:b0640bfe-73f9-4cda-a3ed-720c31e1391d for instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 on hv-cinder-zed-3 Oct 28 09:49:41.000486 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-a53809f3-be63-4a82-b99b-5b6388110b1c req-f014083c-8bf8-484c-9573-ff69b3796e81 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032100 Oct 28 09:49:41.001374 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 321/655] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:49:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:41.333240 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-44beac24-3677-4973-a508-3ee80df86c41 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:41.333884 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-44beac24-3677-4973-a508-3ee80df86c41 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:41.340733 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-44beac24-3677-4973-a508-3ee80df86c41 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:41.341331 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-44beac24-3677-4973-a508-3ee80df86c41 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:41.341772 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-44beac24-3677-4973-a508-3ee80df86c41 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:41.342630 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-44beac24-3677-4973-a508-3ee80df86c41 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:41.343137 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-44beac24-3677-4973-a508-3ee80df86c41 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:41.343633 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-44beac24-3677-4973-a508-3ee80df86c41 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:41.475254 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-44beac24-3677-4973-a508-3ee80df86c41 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:41.482844 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-44beac24-3677-4973-a508-3ee80df86c41 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.154390 Oct 28 09:49:41.483396 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 335/656] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:41 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:41.495067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6b727aae-80d9-4e70-a810-2f4494c4ae1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:41.495796 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6b727aae-80d9-4e70-a810-2f4494c4ae1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:41.503545 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b727aae-80d9-4e70-a810-2f4494c4ae1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:41.504025 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b727aae-80d9-4e70-a810-2f4494c4ae1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:41.504459 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b727aae-80d9-4e70-a810-2f4494c4ae1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:41.505153 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b727aae-80d9-4e70-a810-2f4494c4ae1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:41.505603 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b727aae-80d9-4e70-a810-2f4494c4ae1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:41.506017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6b727aae-80d9-4e70-a810-2f4494c4ae1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:41.645094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6b727aae-80d9-4e70-a810-2f4494c4ae1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:41.653800 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6b727aae-80d9-4e70-a810-2f4494c4ae1a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1724 microversion: 2.1 time: 0.162845 Oct 28 09:49:41.654357 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 322/657] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:41 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1724 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:42.506412 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-4b24ddb7-da2f-4249-b14d-9b55c55dcecd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:42.507035 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-4b24ddb7-da2f-4249-b14d-9b55c55dcecd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:42.513880 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4b24ddb7-da2f-4249-b14d-9b55c55dcecd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:42.514355 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4b24ddb7-da2f-4249-b14d-9b55c55dcecd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:42.514822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4b24ddb7-da2f-4249-b14d-9b55c55dcecd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:42.515508 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4b24ddb7-da2f-4249-b14d-9b55c55dcecd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:42.515954 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4b24ddb7-da2f-4249-b14d-9b55c55dcecd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:42.516371 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4b24ddb7-da2f-4249-b14d-9b55c55dcecd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:42.650517 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-4b24ddb7-da2f-4249-b14d-9b55c55dcecd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:42.659158 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-4b24ddb7-da2f-4249-b14d-9b55c55dcecd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.156643 Oct 28 09:49:42.659758 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 336/658] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:42 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:42.680034 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b3e28efb-c1fc-4081-abf7-0818483a15c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:42.680646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b3e28efb-c1fc-4081-abf7-0818483a15c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:42.687238 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b3e28efb-c1fc-4081-abf7-0818483a15c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:42.687719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b3e28efb-c1fc-4081-abf7-0818483a15c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:42.688153 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b3e28efb-c1fc-4081-abf7-0818483a15c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:42.688849 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b3e28efb-c1fc-4081-abf7-0818483a15c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:42.689296 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b3e28efb-c1fc-4081-abf7-0818483a15c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:42.689719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b3e28efb-c1fc-4081-abf7-0818483a15c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:42.824684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b3e28efb-c1fc-4081-abf7-0818483a15c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:42.833784 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b3e28efb-c1fc-4081-abf7-0818483a15c3 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1724 microversion: 2.1 time: 0.157965 Oct 28 09:49:42.834353 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 323/659] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:42 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1724 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:43.683555 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-3c52ae78-1bb1-492b-964a-02dc8d74eef2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:43.684143 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-3c52ae78-1bb1-492b-964a-02dc8d74eef2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:43.690598 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3c52ae78-1bb1-492b-964a-02dc8d74eef2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:43.691070 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3c52ae78-1bb1-492b-964a-02dc8d74eef2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:43.691497 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3c52ae78-1bb1-492b-964a-02dc8d74eef2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:43.692175 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3c52ae78-1bb1-492b-964a-02dc8d74eef2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:43.692620 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3c52ae78-1bb1-492b-964a-02dc8d74eef2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:43.693031 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-3c52ae78-1bb1-492b-964a-02dc8d74eef2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:43.828365 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-3c52ae78-1bb1-492b-964a-02dc8d74eef2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:43.837871 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-3c52ae78-1bb1-492b-964a-02dc8d74eef2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.158142 Oct 28 09:49:43.838508 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 337/660] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:43 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:43.857026 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-32279c93-1ac0-4d8e-957c-cfcb768266a7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:43.857633 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-32279c93-1ac0-4d8e-957c-cfcb768266a7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:43.864104 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-32279c93-1ac0-4d8e-957c-cfcb768266a7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:43.864581 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-32279c93-1ac0-4d8e-957c-cfcb768266a7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:43.865014 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-32279c93-1ac0-4d8e-957c-cfcb768266a7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:43.865692 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-32279c93-1ac0-4d8e-957c-cfcb768266a7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:43.866138 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-32279c93-1ac0-4d8e-957c-cfcb768266a7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:43.866590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-32279c93-1ac0-4d8e-957c-cfcb768266a7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:43.985788 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6d6d2ae3-d5e5-43ed-96bb-839c0b029ce7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:43.986273 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6d6d2ae3-d5e5-43ed-96bb-839c0b029ce7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:43.988127 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=2 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:43.993586 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6d6d2ae3-d5e5-43ed-96bb-839c0b029ce7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:43.994098 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6d6d2ae3-d5e5-43ed-96bb-839c0b029ce7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:43.994520 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6d6d2ae3-d5e5-43ed-96bb-839c0b029ce7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:43.995208 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6d6d2ae3-d5e5-43ed-96bb-839c0b029ce7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:43.995653 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6d6d2ae3-d5e5-43ed-96bb-839c0b029ce7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:43.996068 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6d6d2ae3-d5e5-43ed-96bb-839c0b029ce7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:43.999987 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-32279c93-1ac0-4d8e-957c-cfcb768266a7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:44.000150 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=100 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:44.010720 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-32279c93-1ac0-4d8e-957c-cfcb768266a7 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1724 microversion: 2.1 time: 0.157332 Oct 28 09:49:44.011631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 324/661] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:43 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1724 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:44.012351 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=10 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:44.030738 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6d6d2ae3-d5e5-43ed-96bb-839c0b029ce7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments/a18cc270-0f5e-4a69-9502-54ad13534acb" status: 200 len: 194 microversion: 2.20 time: 0.048735 Oct 28 09:49:44.031284 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 338/662] 10.222.0.25 () {60 vars in 1454 bytes} [Fri Oct 28 09:49:43 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments/a18cc270-0f5e-4a69-9502-54ad13534acb => generated 194 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 28 09:49:44.107024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8dd4ac25-9d6b-4aa0-820a-532657e32a37 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:44.107538 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8dd4ac25-9d6b-4aa0-820a-532657e32a37 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:44.113615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8dd4ac25-9d6b-4aa0-820a-532657e32a37 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:44.114086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8dd4ac25-9d6b-4aa0-820a-532657e32a37 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:44.114543 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8dd4ac25-9d6b-4aa0-820a-532657e32a37 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:44.115224 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8dd4ac25-9d6b-4aa0-820a-532657e32a37 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:44.115662 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8dd4ac25-9d6b-4aa0-820a-532657e32a37 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:44.116090 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8dd4ac25-9d6b-4aa0-820a-532657e32a37 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:44.146600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:44.198029 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-8dd4ac25-9d6b-4aa0-820a-532657e32a37 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lazy-loading 'services' on Instance uuid c716d5fd-f257-4f6a-8f52-18ad59e5c31e {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:49:44.273629 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8dd4ac25-9d6b-4aa0-820a-532657e32a37 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments/a18cc270-0f5e-4a69-9502-54ad13534acb" status: 202 len: 0 microversion: 2.20 time: 0.170399 Oct 28 09:49:44.274226 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 325/663] 10.222.0.25 () {60 vars in 1457 bytes} [Fri Oct 28 09:49:44 2022] DELETE /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments/a18cc270-0f5e-4a69-9502-54ad13534acb => generated 0 bytes in 172 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:49:44.283161 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-7ae144e9-c00f-4d80-a237-bca305721225 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:44.285355 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-7ae144e9-c00f-4d80-a237-bca305721225 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:44.291417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7ae144e9-c00f-4d80-a237-bca305721225 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:44.291897 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7ae144e9-c00f-4d80-a237-bca305721225 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:44.292329 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7ae144e9-c00f-4d80-a237-bca305721225 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:44.293012 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7ae144e9-c00f-4d80-a237-bca305721225 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:44.293459 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7ae144e9-c00f-4d80-a237-bca305721225 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:44.293879 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7ae144e9-c00f-4d80-a237-bca305721225 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:44.324880 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-7ae144e9-c00f-4d80-a237-bca305721225 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045888 Oct 28 09:49:44.325473 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 339/664] 10.222.0.25 () {60 vars in 1343 bytes} [Fri Oct 28 09:49:44 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 28 09:49:44.867947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8cc491ee-4317-4d02-8e66-bf27f1bc9a30 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:44.869240 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8cc491ee-4317-4d02-8e66-bf27f1bc9a30 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:44.879386 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8cc491ee-4317-4d02-8e66-bf27f1bc9a30 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:44.879926 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8cc491ee-4317-4d02-8e66-bf27f1bc9a30 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:44.880467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8cc491ee-4317-4d02-8e66-bf27f1bc9a30 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:44.881322 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8cc491ee-4317-4d02-8e66-bf27f1bc9a30 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:44.881910 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8cc491ee-4317-4d02-8e66-bf27f1bc9a30 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:44.882654 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8cc491ee-4317-4d02-8e66-bf27f1bc9a30 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:45.029314 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-25f4ebdc-3924-4647-b293-25ac17248e08 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-907287077"}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:45.029314 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-25f4ebdc-3924-4647-b293-25ac17248e08 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:45.031081 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8cc491ee-4317-4d02-8e66-bf27f1bc9a30 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:45.037742 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-25f4ebdc-3924-4647-b293-25ac17248e08 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:45.038266 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-25f4ebdc-3924-4647-b293-25ac17248e08 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:45.038937 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-25f4ebdc-3924-4647-b293-25ac17248e08 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:45.040134 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-25f4ebdc-3924-4647-b293-25ac17248e08 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:45.040835 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-25f4ebdc-3924-4647-b293-25ac17248e08 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:45.041274 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8cc491ee-4317-4d02-8e66-bf27f1bc9a30 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.182462 Oct 28 09:49:45.041591 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-25f4ebdc-3924-4647-b293-25ac17248e08 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:45.042012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 326/665] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:44 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:45.046876 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-dac6cc65-9f95-49b7-8b08-1714e3ae7bd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:45.047708 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-dac6cc65-9f95-49b7-8b08-1714e3ae7bd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:45.055720 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dac6cc65-9f95-49b7-8b08-1714e3ae7bd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:45.056418 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dac6cc65-9f95-49b7-8b08-1714e3ae7bd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:45.057145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dac6cc65-9f95-49b7-8b08-1714e3ae7bd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:45.058036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dac6cc65-9f95-49b7-8b08-1714e3ae7bd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:45.058664 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dac6cc65-9f95-49b7-8b08-1714e3ae7bd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:45.059254 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dac6cc65-9f95-49b7-8b08-1714e3ae7bd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:45.256263 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.network.security_group_api [None req-25f4ebdc-3924-4647-b293-25ac17248e08 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Removing security group b42bf8e7-036b-494e-937c-9d6eeea54ddc from port 0ec06b02-a12d-496d-a40d-c56118810a83 Oct 28 09:49:45.262931 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-dac6cc65-9f95-49b7-8b08-1714e3ae7bd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:45.272998 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-dac6cc65-9f95-49b7-8b08-1714e3ae7bd2 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1724 microversion: 2.1 time: 0.229897 Oct 28 09:49:45.289915 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 327/666] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:45 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1724 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:45.340993 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-995d023e-c60e-410e-b781-5c81b2191e9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:45.343633 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-995d023e-c60e-410e-b781-5c81b2191e9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:45.353092 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-995d023e-c60e-410e-b781-5c81b2191e9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:45.353586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-995d023e-c60e-410e-b781-5c81b2191e9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:45.354022 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-995d023e-c60e-410e-b781-5c81b2191e9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:45.354780 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-995d023e-c60e-410e-b781-5c81b2191e9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:45.355272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-995d023e-c60e-410e-b781-5c81b2191e9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:45.355647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-995d023e-c60e-410e-b781-5c81b2191e9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:45.397221 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-995d023e-c60e-410e-b781-5c81b2191e9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061516 Oct 28 09:49:45.397737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 328/667] 10.222.0.25 () {60 vars in 1343 bytes} [Fri Oct 28 09:49:45 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:49:45.631669 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-25f4ebdc-3924-4647-b293-25ac17248e08 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "POST /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/action" status: 202 len: 0 microversion: 2.1 time: 0.610074 Oct 28 09:49:45.632445 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 340/668] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:49:45 2022] POST /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/action => generated 0 bytes in 612 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:49:45.652090 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-ab034336-a8a5-4286-879d-76a72648b64f req-301dcf12-35ef-48dd-8e12-b3db5f3a9879 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "04eedf41-b3f2-4b33-b524-988daff99fdc", "tag": "0ec06b02-a12d-496d-a40d-c56118810a83"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:45.684397 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-ab034336-a8a5-4286-879d-76a72648b64f req-301dcf12-35ef-48dd-8e12-b3db5f3a9879 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:45.686684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-ab034336-a8a5-4286-879d-76a72648b64f req-301dcf12-35ef-48dd-8e12-b3db5f3a9879 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:45.687152 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-ab034336-a8a5-4286-879d-76a72648b64f req-301dcf12-35ef-48dd-8e12-b3db5f3a9879 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:45.711117 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-ab034336-a8a5-4286-879d-76a72648b64f req-301dcf12-35ef-48dd-8e12-b3db5f3a9879 service nova] Creating event network-changed:0ec06b02-a12d-496d-a40d-c56118810a83 for instance 04eedf41-b3f2-4b33-b524-988daff99fdc on hv-cinder-zed-3 Oct 28 09:49:45.717737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-ab034336-a8a5-4286-879d-76a72648b64f req-301dcf12-35ef-48dd-8e12-b3db5f3a9879 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.079336 Oct 28 09:49:45.719072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 329/669] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:49:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:46.066135 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-062d1f55-ede5-4a6a-bc20-ae2167d9e388 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:46.066871 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-062d1f55-ede5-4a6a-bc20-ae2167d9e388 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:46.072961 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-062d1f55-ede5-4a6a-bc20-ae2167d9e388 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:46.073445 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-062d1f55-ede5-4a6a-bc20-ae2167d9e388 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:46.073885 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-062d1f55-ede5-4a6a-bc20-ae2167d9e388 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:46.074624 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-062d1f55-ede5-4a6a-bc20-ae2167d9e388 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:46.075062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-062d1f55-ede5-4a6a-bc20-ae2167d9e388 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:46.075487 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-062d1f55-ede5-4a6a-bc20-ae2167d9e388 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:46.298824 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6d5b5909-7776-47f0-bf7f-31f9d60dd174 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:46.299622 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6d5b5909-7776-47f0-bf7f-31f9d60dd174 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:46.301688 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=98,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:46.311930 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d5b5909-7776-47f0-bf7f-31f9d60dd174 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:46.312645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d5b5909-7776-47f0-bf7f-31f9d60dd174 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:46.313243 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d5b5909-7776-47f0-bf7f-31f9d60dd174 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:46.314456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d5b5909-7776-47f0-bf7f-31f9d60dd174 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:46.315240 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d5b5909-7776-47f0-bf7f-31f9d60dd174 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:46.315683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d5b5909-7776-47f0-bf7f-31f9d60dd174 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:46.530032 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-062d1f55-ede5-4a6a-bc20-ae2167d9e388 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:46.549384 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-062d1f55-ede5-4a6a-bc20-ae2167d9e388 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.487634 Oct 28 09:49:46.554089 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 341/670] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:46 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:46.561132 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-7f1f1fa1-afa6-42a2-9042-951f6c486841 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:46.565595 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-7f1f1fa1-afa6-42a2-9042-951f6c486841 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:46.572284 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7f1f1fa1-afa6-42a2-9042-951f6c486841 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:46.573063 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7f1f1fa1-afa6-42a2-9042-951f6c486841 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:46.573824 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7f1f1fa1-afa6-42a2-9042-951f6c486841 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:46.574982 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7f1f1fa1-afa6-42a2-9042-951f6c486841 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:46.575626 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7f1f1fa1-afa6-42a2-9042-951f6c486841 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:46.576258 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7f1f1fa1-afa6-42a2-9042-951f6c486841 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:46.614453 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-7f1f1fa1-afa6-42a2-9042-951f6c486841 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058306 Oct 28 09:49:46.615397 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 342/671] 10.222.0.25 () {60 vars in 1343 bytes} [Fri Oct 28 09:49:46 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:49:46.773300 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6d5b5909-7776-47f0-bf7f-31f9d60dd174 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:46.782011 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6d5b5909-7776-47f0-bf7f-31f9d60dd174 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1730 microversion: 2.1 time: 0.488114 Oct 28 09:49:46.782646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 330/672] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:46 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1730 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:46.806306 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-62c9a7de-d454-46ce-a4bd-1dc6bfbc0d6e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:46.807049 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-62c9a7de-d454-46ce-a4bd-1dc6bfbc0d6e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:46.813042 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-62c9a7de-d454-46ce-a4bd-1dc6bfbc0d6e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:46.813499 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-62c9a7de-d454-46ce-a4bd-1dc6bfbc0d6e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:46.814538 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-62c9a7de-d454-46ce-a4bd-1dc6bfbc0d6e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:46.815249 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-62c9a7de-d454-46ce-a4bd-1dc6bfbc0d6e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:46.815693 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-62c9a7de-d454-46ce-a4bd-1dc6bfbc0d6e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:46.816116 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-62c9a7de-d454-46ce-a4bd-1dc6bfbc0d6e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:47.022802 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-62c9a7de-d454-46ce-a4bd-1dc6bfbc0d6e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.220168 Oct 28 09:49:47.023600 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 343/673] 10.222.0.25 () {58 vars in 1275 bytes} [Fri Oct 28 09:49:46 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-interface => generated 284 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:47.581465 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7924cbfd-0ffa-47a5-85c9-3414f91d0a40 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:47.582080 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7924cbfd-0ffa-47a5-85c9-3414f91d0a40 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:47.590449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7924cbfd-0ffa-47a5-85c9-3414f91d0a40 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:47.591413 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7924cbfd-0ffa-47a5-85c9-3414f91d0a40 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:47.592032 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7924cbfd-0ffa-47a5-85c9-3414f91d0a40 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:47.592754 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7924cbfd-0ffa-47a5-85c9-3414f91d0a40 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:47.593203 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7924cbfd-0ffa-47a5-85c9-3414f91d0a40 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:47.593617 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7924cbfd-0ffa-47a5-85c9-3414f91d0a40 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:47.628478 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6ed1fbf7-7569-413b-b4e5-1375c9e896db tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:47.632050 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6ed1fbf7-7569-413b-b4e5-1375c9e896db tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:47.644566 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6ed1fbf7-7569-413b-b4e5-1375c9e896db tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:47.645224 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6ed1fbf7-7569-413b-b4e5-1375c9e896db tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:47.645661 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6ed1fbf7-7569-413b-b4e5-1375c9e896db tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:47.646361 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6ed1fbf7-7569-413b-b4e5-1375c9e896db tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:47.647011 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6ed1fbf7-7569-413b-b4e5-1375c9e896db tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:47.647486 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6ed1fbf7-7569-413b-b4e5-1375c9e896db tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:47.694707 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6ed1fbf7-7569-413b-b4e5-1375c9e896db tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.070459 Oct 28 09:49:47.696104 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 344/674] 10.222.0.25 () {60 vars in 1343 bytes} [Fri Oct 28 09:49:47 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e/os-volume_attachments => generated 25 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:47.801080 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8b0912a5-8ec9-42ca-977a-061e5cf371d7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:47.801740 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8b0912a5-8ec9-42ca-977a-061e5cf371d7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:47.810305 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8b0912a5-8ec9-42ca-977a-061e5cf371d7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:47.811181 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8b0912a5-8ec9-42ca-977a-061e5cf371d7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:47.812570 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8b0912a5-8ec9-42ca-977a-061e5cf371d7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:47.814135 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8b0912a5-8ec9-42ca-977a-061e5cf371d7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:47.815296 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8b0912a5-8ec9-42ca-977a-061e5cf371d7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:47.816214 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8b0912a5-8ec9-42ca-977a-061e5cf371d7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:47.850175 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8b0912a5-8ec9-42ca-977a-061e5cf371d7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Going to try to terminate instance {{(pid=127029) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:49:47.996543 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7924cbfd-0ffa-47a5-85c9-3414f91d0a40 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:48.008592 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7924cbfd-0ffa-47a5-85c9-3414f91d0a40 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.432026 Oct 28 09:49:48.022841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 331/675] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:47 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:48.055043 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8b0912a5-8ec9-42ca-977a-061e5cf371d7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 204 len: 0 microversion: 2.20 time: 0.255153 Oct 28 09:49:48.063291 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 345/676] 10.222.0.25 () {60 vars in 1280 bytes} [Fri Oct 28 09:49:47 2022] DELETE /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 28 09:49:48.068515 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-eca10cdf-4567-45c4-851b-b574b4db6654 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:48.068873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-eca10cdf-4567-45c4-851b-b574b4db6654 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:48.079967 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eca10cdf-4567-45c4-851b-b574b4db6654 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:48.080768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eca10cdf-4567-45c4-851b-b574b4db6654 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:48.081449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eca10cdf-4567-45c4-851b-b574b4db6654 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:48.085571 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eca10cdf-4567-45c4-851b-b574b4db6654 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:48.086360 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eca10cdf-4567-45c4-851b-b574b4db6654 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:48.087150 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eca10cdf-4567-45c4-851b-b574b4db6654 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:48.087844 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-fac00cd9-4cd0-4b4d-be7a-3dea4d96d3d0 req-79515c0d-003c-455b-a73f-e4d99abaae06 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4891b63b-eb08-448d-a640-f9579c8140ff", "tag": "6e78f30d-6812-43c6-910d-70f563ee8ad9"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:48.111689 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-fac00cd9-4cd0-4b4d-be7a-3dea4d96d3d0 req-79515c0d-003c-455b-a73f-e4d99abaae06 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:48.112427 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-fac00cd9-4cd0-4b4d-be7a-3dea4d96d3d0 req-79515c0d-003c-455b-a73f-e4d99abaae06 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:48.113050 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-fac00cd9-4cd0-4b4d-be7a-3dea4d96d3d0 req-79515c0d-003c-455b-a73f-e4d99abaae06 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:48.153462 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-fac00cd9-4cd0-4b4d-be7a-3dea4d96d3d0 req-79515c0d-003c-455b-a73f-e4d99abaae06 service nova] Creating event network-changed:6e78f30d-6812-43c6-910d-70f563ee8ad9 for instance 4891b63b-eb08-448d-a640-f9579c8140ff on hv-cinder-zed-3 Oct 28 09:49:48.159606 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-fac00cd9-4cd0-4b4d-be7a-3dea4d96d3d0 req-79515c0d-003c-455b-a73f-e4d99abaae06 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.075935 Oct 28 09:49:48.169954 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 346/677] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:49:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:48.219303 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-616529db-71fe-460b-8e80-2f49030d7e24 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:48.219932 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-616529db-71fe-460b-8e80-2f49030d7e24 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:48.238775 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-616529db-71fe-460b-8e80-2f49030d7e24 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:48.239979 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-616529db-71fe-460b-8e80-2f49030d7e24 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:48.240459 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-616529db-71fe-460b-8e80-2f49030d7e24 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:48.241309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-616529db-71fe-460b-8e80-2f49030d7e24 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:48.242026 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-616529db-71fe-460b-8e80-2f49030d7e24 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:48.242478 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-616529db-71fe-460b-8e80-2f49030d7e24 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:48.304822 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-eca10cdf-4567-45c4-851b-b574b4db6654 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:48.313941 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-eca10cdf-4567-45c4-851b-b574b4db6654 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:48.316046 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-eca10cdf-4567-45c4-851b-b574b4db6654 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:48.317493 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-eca10cdf-4567-45c4-851b-b574b4db6654 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.254209 Oct 28 09:49:48.318640 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 332/678] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:49:48 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:49:48.323696 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-39f22bda-b173-44ae-b70b-403c671c77ea req-5e267196-4bf9-45d7-8914-774c2f2027ed service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "04eedf41-b3f2-4b33-b524-988daff99fdc", "tag": "0ec06b02-a12d-496d-a40d-c56118810a83"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:48.337096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-39f22bda-b173-44ae-b70b-403c671c77ea req-5e267196-4bf9-45d7-8914-774c2f2027ed service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:48.338249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-39f22bda-b173-44ae-b70b-403c671c77ea req-5e267196-4bf9-45d7-8914-774c2f2027ed service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:48.339172 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-39f22bda-b173-44ae-b70b-403c671c77ea req-5e267196-4bf9-45d7-8914-774c2f2027ed service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:48.352612 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-39f22bda-b173-44ae-b70b-403c671c77ea req-5e267196-4bf9-45d7-8914-774c2f2027ed service nova] Creating event network-changed:0ec06b02-a12d-496d-a40d-c56118810a83 for instance 04eedf41-b3f2-4b33-b524-988daff99fdc on hv-cinder-zed-3 Oct 28 09:49:48.358667 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-39f22bda-b173-44ae-b70b-403c671c77ea req-5e267196-4bf9-45d7-8914-774c2f2027ed service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038302 Oct 28 09:49:48.360008 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 333/679] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:49:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:48.375301 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.objects.instance [None req-616529db-71fe-460b-8e80-2f49030d7e24 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lazy-loading 'services' on Instance uuid 04eedf41-b3f2-4b33-b524-988daff99fdc {{(pid=127029) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:49:48.459283 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-616529db-71fe-460b-8e80-2f49030d7e24 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "DELETE /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments/4bfce979-c3b3-4e9d-b857-326102f2c9ca" status: 202 len: 0 microversion: 2.1 time: 0.244358 Oct 28 09:49:48.459875 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 347/680] 10.222.0.25 () {58 vars in 1416 bytes} [Fri Oct 28 09:49:48 2022] DELETE /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments/4bfce979-c3b3-4e9d-b857-326102f2c9ca => generated 0 bytes in 247 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 09:49:48.469625 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9709f6e4-905e-4691-bd9c-d49ef7ac6e89 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:48.472925 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9709f6e4-905e-4691-bd9c-d49ef7ac6e89 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:48.479602 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9709f6e4-905e-4691-bd9c-d49ef7ac6e89 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:48.480395 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9709f6e4-905e-4691-bd9c-d49ef7ac6e89 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:48.480836 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9709f6e4-905e-4691-bd9c-d49ef7ac6e89 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:48.481621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9709f6e4-905e-4691-bd9c-d49ef7ac6e89 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:48.482267 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9709f6e4-905e-4691-bd9c-d49ef7ac6e89 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:48.482899 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9709f6e4-905e-4691-bd9c-d49ef7ac6e89 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:48.501437 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:48.520344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9709f6e4-905e-4691-bd9c-d49ef7ac6e89 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055200 Oct 28 09:49:48.521227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 334/681] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:49:48 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:49.040637 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b10366ab-a1ab-493e-8173-f5c1727d0a97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:49.042811 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b10366ab-a1ab-493e-8173-f5c1727d0a97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:49.055224 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b10366ab-a1ab-493e-8173-f5c1727d0a97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:49.056087 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b10366ab-a1ab-493e-8173-f5c1727d0a97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:49.056865 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b10366ab-a1ab-493e-8173-f5c1727d0a97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:49.057864 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b10366ab-a1ab-493e-8173-f5c1727d0a97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:49.058925 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b10366ab-a1ab-493e-8173-f5c1727d0a97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:49.059898 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b10366ab-a1ab-493e-8173-f5c1727d0a97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:49.228932 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b10366ab-a1ab-493e-8173-f5c1727d0a97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:49.239958 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b10366ab-a1ab-493e-8173-f5c1727d0a97 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.207072 Oct 28 09:49:49.240575 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 348/682] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:49 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:49.348771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2ae32883-d521-4a05-aa9a-3937e51b23bc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:49.349992 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2ae32883-d521-4a05-aa9a-3937e51b23bc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:49.357245 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ae32883-d521-4a05-aa9a-3937e51b23bc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:49.357748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ae32883-d521-4a05-aa9a-3937e51b23bc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:49.358178 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ae32883-d521-4a05-aa9a-3937e51b23bc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:49.359161 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ae32883-d521-4a05-aa9a-3937e51b23bc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:49.359643 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ae32883-d521-4a05-aa9a-3937e51b23bc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:49.360033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ae32883-d521-4a05-aa9a-3937e51b23bc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:49.534907 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-80f2375d-36fd-49eb-b9be-94bfb07c0232 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:49.538276 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-80f2375d-36fd-49eb-b9be-94bfb07c0232 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:49.546076 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-80f2375d-36fd-49eb-b9be-94bfb07c0232 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:49.546650 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-80f2375d-36fd-49eb-b9be-94bfb07c0232 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:49.547082 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-80f2375d-36fd-49eb-b9be-94bfb07c0232 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:49.547819 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-80f2375d-36fd-49eb-b9be-94bfb07c0232 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:49.548333 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-80f2375d-36fd-49eb-b9be-94bfb07c0232 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:49.548466 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2ae32883-d521-4a05-aa9a-3937e51b23bc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:49.548751 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-80f2375d-36fd-49eb-b9be-94bfb07c0232 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:49.563387 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2ae32883-d521-4a05-aa9a-3937e51b23bc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:49.565348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2ae32883-d521-4a05-aa9a-3937e51b23bc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:49.566773 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2ae32883-d521-4a05-aa9a-3937e51b23bc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.223126 Oct 28 09:49:49.576413 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 335/683] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:49:49 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:49:49.589530 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-80f2375d-36fd-49eb-b9be-94bfb07c0232 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059464 Oct 28 09:49:49.590518 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 349/684] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:49:49 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:50.176968 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-c45fbbd2-dfcc-4a10-bb69-9364cbebbfef req-072a06a6-09fc-40d5-a1be-db17c41593e4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4891b63b-eb08-448d-a640-f9579c8140ff", "tag": "6e78f30d-6812-43c6-910d-70f563ee8ad9"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:50.186978 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-c45fbbd2-dfcc-4a10-bb69-9364cbebbfef req-072a06a6-09fc-40d5-a1be-db17c41593e4 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:50.187454 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-c45fbbd2-dfcc-4a10-bb69-9364cbebbfef req-072a06a6-09fc-40d5-a1be-db17c41593e4 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:50.187880 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-c45fbbd2-dfcc-4a10-bb69-9364cbebbfef req-072a06a6-09fc-40d5-a1be-db17c41593e4 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:50.199678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-c45fbbd2-dfcc-4a10-bb69-9364cbebbfef req-072a06a6-09fc-40d5-a1be-db17c41593e4 service nova] Creating event network-changed:6e78f30d-6812-43c6-910d-70f563ee8ad9 for instance 4891b63b-eb08-448d-a640-f9579c8140ff on hv-cinder-zed-3 Oct 28 09:49:50.205070 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-c45fbbd2-dfcc-4a10-bb69-9364cbebbfef req-072a06a6-09fc-40d5-a1be-db17c41593e4 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032111 Oct 28 09:49:50.205677 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 336/685] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:49:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:50.263259 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-443a7480-c944-489c-a6b0-03c5f6fa7588 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:50.263855 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-443a7480-c944-489c-a6b0-03c5f6fa7588 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:50.270685 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-443a7480-c944-489c-a6b0-03c5f6fa7588 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:50.271160 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-443a7480-c944-489c-a6b0-03c5f6fa7588 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:50.271588 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-443a7480-c944-489c-a6b0-03c5f6fa7588 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:50.272273 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-443a7480-c944-489c-a6b0-03c5f6fa7588 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:50.272780 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-443a7480-c944-489c-a6b0-03c5f6fa7588 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:50.273129 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-443a7480-c944-489c-a6b0-03c5f6fa7588 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:50.405691 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-443a7480-c944-489c-a6b0-03c5f6fa7588 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:50.413977 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-443a7480-c944-489c-a6b0-03c5f6fa7588 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.154598 Oct 28 09:49:50.414566 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 350/686] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:50 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:50.593501 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-640aa49e-141d-4cb2-8a5d-f33617f926ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:50.594216 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-640aa49e-141d-4cb2-8a5d-f33617f926ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:50.601333 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-640aa49e-141d-4cb2-8a5d-f33617f926ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:50.601748 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-16306f46-c661-43ee-bed6-383bfa2b8219 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:50.601881 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-640aa49e-141d-4cb2-8a5d-f33617f926ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:50.602219 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-640aa49e-141d-4cb2-8a5d-f33617f926ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:50.602949 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-640aa49e-141d-4cb2-8a5d-f33617f926ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:50.603392 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-640aa49e-141d-4cb2-8a5d-f33617f926ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:50.603805 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-640aa49e-141d-4cb2-8a5d-f33617f926ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:50.603916 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-16306f46-c661-43ee-bed6-383bfa2b8219 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:50.610253 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-16306f46-c661-43ee-bed6-383bfa2b8219 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:50.610788 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-16306f46-c661-43ee-bed6-383bfa2b8219 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:50.611309 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-16306f46-c661-43ee-bed6-383bfa2b8219 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:50.611898 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-16306f46-c661-43ee-bed6-383bfa2b8219 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:50.612337 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-16306f46-c661-43ee-bed6-383bfa2b8219 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:50.612749 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-16306f46-c661-43ee-bed6-383bfa2b8219 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:50.643066 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-16306f46-c661-43ee-bed6-383bfa2b8219 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045144 Oct 28 09:49:50.644072 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 351/687] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:49:50 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:49:50.735447 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-640aa49e-141d-4cb2-8a5d-f33617f926ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:50.743711 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-640aa49e-141d-4cb2-8a5d-f33617f926ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:50.745140 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-640aa49e-141d-4cb2-8a5d-f33617f926ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:50.746288 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-640aa49e-141d-4cb2-8a5d-f33617f926ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.157479 Oct 28 09:49:50.746877 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 337/688] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:49:50 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:49:51.437315 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-9d6b1b15-a460-43c2-a66c-3bc52865039b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:51.437898 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-9d6b1b15-a460-43c2-a66c-3bc52865039b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:51.444465 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9d6b1b15-a460-43c2-a66c-3bc52865039b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:51.444963 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9d6b1b15-a460-43c2-a66c-3bc52865039b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:51.445390 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9d6b1b15-a460-43c2-a66c-3bc52865039b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:51.446067 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9d6b1b15-a460-43c2-a66c-3bc52865039b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:51.446606 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9d6b1b15-a460-43c2-a66c-3bc52865039b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:51.447023 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-9d6b1b15-a460-43c2-a66c-3bc52865039b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:51.581275 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-9d6b1b15-a460-43c2-a66c-3bc52865039b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:51.589312 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-9d6b1b15-a460-43c2-a66c-3bc52865039b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.155933 Oct 28 09:49:51.589921 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 352/689] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:51 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:51.655444 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6454cb1d-234b-4957-9f66-ba6e8d62bd85 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:51.658092 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6454cb1d-234b-4957-9f66-ba6e8d62bd85 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:51.664923 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6454cb1d-234b-4957-9f66-ba6e8d62bd85 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:51.665391 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6454cb1d-234b-4957-9f66-ba6e8d62bd85 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:51.665816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6454cb1d-234b-4957-9f66-ba6e8d62bd85 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:51.666611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6454cb1d-234b-4957-9f66-ba6e8d62bd85 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:51.667062 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6454cb1d-234b-4957-9f66-ba6e8d62bd85 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:51.667476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6454cb1d-234b-4957-9f66-ba6e8d62bd85 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:51.698681 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6454cb1d-234b-4957-9f66-ba6e8d62bd85 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047042 Oct 28 09:49:51.699165 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 338/690] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:49:51 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:51.772206 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-00080daf-62ec-4df0-9663-442399856ff3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:51.772831 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-00080daf-62ec-4df0-9663-442399856ff3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:51.779046 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-00080daf-62ec-4df0-9663-442399856ff3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:51.779524 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-00080daf-62ec-4df0-9663-442399856ff3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:51.779952 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-00080daf-62ec-4df0-9663-442399856ff3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:51.780639 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-00080daf-62ec-4df0-9663-442399856ff3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:51.781201 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-00080daf-62ec-4df0-9663-442399856ff3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:51.781616 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-00080daf-62ec-4df0-9663-442399856ff3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:51.914744 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-00080daf-62ec-4df0-9663-442399856ff3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:51.918332 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:51.924615 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-00080daf-62ec-4df0-9663-442399856ff3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:51.926067 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-00080daf-62ec-4df0-9663-442399856ff3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:51.927264 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-00080daf-62ec-4df0-9663-442399856ff3 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.159003 Oct 28 09:49:51.927818 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 353/691] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:49:51 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:49:52.612771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-10896d5a-fe71-4538-ba5c-20c1a03b051a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:52.613381 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-10896d5a-fe71-4538-ba5c-20c1a03b051a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:52.619533 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-10896d5a-fe71-4538-ba5c-20c1a03b051a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:52.620018 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-10896d5a-fe71-4538-ba5c-20c1a03b051a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:52.620905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-10896d5a-fe71-4538-ba5c-20c1a03b051a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:52.621140 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-10896d5a-fe71-4538-ba5c-20c1a03b051a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:52.621587 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-10896d5a-fe71-4538-ba5c-20c1a03b051a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:52.622010 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-10896d5a-fe71-4538-ba5c-20c1a03b051a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:52.712948 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e378ebbe-0cb5-42b3-935a-a063b771189e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:52.715262 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e378ebbe-0cb5-42b3-935a-a063b771189e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:52.723694 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e378ebbe-0cb5-42b3-935a-a063b771189e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:52.724220 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e378ebbe-0cb5-42b3-935a-a063b771189e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:52.724623 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e378ebbe-0cb5-42b3-935a-a063b771189e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:52.725329 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e378ebbe-0cb5-42b3-935a-a063b771189e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:52.725780 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e378ebbe-0cb5-42b3-935a-a063b771189e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:52.726208 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e378ebbe-0cb5-42b3-935a-a063b771189e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:52.759358 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e378ebbe-0cb5-42b3-935a-a063b771189e tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051872 Oct 28 09:49:52.759957 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 354/692] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:49:52 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:49:52.763557 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-10896d5a-fe71-4538-ba5c-20c1a03b051a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:52.771813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-10896d5a-fe71-4538-ba5c-20c1a03b051a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.162841 Oct 28 09:49:52.772307 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 339/693] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:52 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:52.952485 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-03346f9e-10c8-423d-be20-a60a8d23042f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:52.953108 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-03346f9e-10c8-423d-be20-a60a8d23042f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:52.959699 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-03346f9e-10c8-423d-be20-a60a8d23042f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:52.960172 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-03346f9e-10c8-423d-be20-a60a8d23042f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:52.960596 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-03346f9e-10c8-423d-be20-a60a8d23042f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:52.961271 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-03346f9e-10c8-423d-be20-a60a8d23042f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:52.961708 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-03346f9e-10c8-423d-be20-a60a8d23042f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:52.962122 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-03346f9e-10c8-423d-be20-a60a8d23042f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:53.095587 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-03346f9e-10c8-423d-be20-a60a8d23042f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:53.103845 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-03346f9e-10c8-423d-be20-a60a8d23042f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:53.105298 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-03346f9e-10c8-423d-be20-a60a8d23042f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:53.106491 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-03346f9e-10c8-423d-be20-a60a8d23042f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.157892 Oct 28 09:49:53.107062 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 355/694] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:49:52 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:49:53.772365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-05846436-4b83-49b9-b7e1-42eef3d4ee74 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:53.774516 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-05846436-4b83-49b9-b7e1-42eef3d4ee74 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:53.780953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-05846436-4b83-49b9-b7e1-42eef3d4ee74 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:53.781429 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-05846436-4b83-49b9-b7e1-42eef3d4ee74 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:53.781887 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-05846436-4b83-49b9-b7e1-42eef3d4ee74 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:53.783098 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-05846436-4b83-49b9-b7e1-42eef3d4ee74 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:53.783541 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-05846436-4b83-49b9-b7e1-42eef3d4ee74 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:53.783954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-05846436-4b83-49b9-b7e1-42eef3d4ee74 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:53.796681 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-fbd2d190-6132-4856-b060-ad543407a06d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:53.797505 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-fbd2d190-6132-4856-b060-ad543407a06d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:53.804985 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fbd2d190-6132-4856-b060-ad543407a06d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:53.805647 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fbd2d190-6132-4856-b060-ad543407a06d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:53.806069 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fbd2d190-6132-4856-b060-ad543407a06d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:53.806942 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fbd2d190-6132-4856-b060-ad543407a06d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:53.807537 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fbd2d190-6132-4856-b060-ad543407a06d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:53.808106 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fbd2d190-6132-4856-b060-ad543407a06d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:53.820357 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-05846436-4b83-49b9-b7e1-42eef3d4ee74 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051755 Oct 28 09:49:53.820939 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 340/695] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:49:53 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:53.967252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-fbd2d190-6132-4856-b060-ad543407a06d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:53.978423 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-fbd2d190-6132-4856-b060-ad543407a06d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.186323 Oct 28 09:49:53.979340 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 356/696] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:53 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:54.133070 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0402010b-f462-47de-b6e1-a292fdf00e62 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:54.133667 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0402010b-f462-47de-b6e1-a292fdf00e62 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:54.139875 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0402010b-f462-47de-b6e1-a292fdf00e62 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:54.140541 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0402010b-f462-47de-b6e1-a292fdf00e62 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:54.141086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0402010b-f462-47de-b6e1-a292fdf00e62 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:54.142345 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0402010b-f462-47de-b6e1-a292fdf00e62 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:54.142947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0402010b-f462-47de-b6e1-a292fdf00e62 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:54.143378 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0402010b-f462-47de-b6e1-a292fdf00e62 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:54.668101 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0402010b-f462-47de-b6e1-a292fdf00e62 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:54.677317 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0402010b-f462-47de-b6e1-a292fdf00e62 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:54.678819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0402010b-f462-47de-b6e1-a292fdf00e62 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:54.680250 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0402010b-f462-47de-b6e1-a292fdf00e62 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.551163 Oct 28 09:49:54.690453 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 341/697] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:49:54 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:49:54.834527 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-884da0b9-5df0-4c00-8c6a-048950b519ac tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:54.836838 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-884da0b9-5df0-4c00-8c6a-048950b519ac tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:54.843692 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-884da0b9-5df0-4c00-8c6a-048950b519ac tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:54.844176 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-884da0b9-5df0-4c00-8c6a-048950b519ac tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:54.844614 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-884da0b9-5df0-4c00-8c6a-048950b519ac tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:54.845750 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-884da0b9-5df0-4c00-8c6a-048950b519ac tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:54.846339 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-884da0b9-5df0-4c00-8c6a-048950b519ac tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:54.846791 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-884da0b9-5df0-4c00-8c6a-048950b519ac tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:54.880166 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-884da0b9-5df0-4c00-8c6a-048950b519ac tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.050491 Oct 28 09:49:54.880747 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 357/698] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:49:54 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 09:49:54.892871 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1c194e85-f165-4c29-afcc-9c6fb49dda22 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:54.893936 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1c194e85-f165-4c29-afcc-9c6fb49dda22 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:54.900547 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1c194e85-f165-4c29-afcc-9c6fb49dda22 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:54.901032 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1c194e85-f165-4c29-afcc-9c6fb49dda22 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:54.901467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1c194e85-f165-4c29-afcc-9c6fb49dda22 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:54.902166 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1c194e85-f165-4c29-afcc-9c6fb49dda22 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:54.902702 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1c194e85-f165-4c29-afcc-9c6fb49dda22 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:54.903398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1c194e85-f165-4c29-afcc-9c6fb49dda22 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:55.007691 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b0c7f50b-42e0-4e42-ac68-0cbb6cd9293b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:55.009138 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b0c7f50b-42e0-4e42-ac68-0cbb6cd9293b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:55.019866 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b0c7f50b-42e0-4e42-ac68-0cbb6cd9293b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:55.021544 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b0c7f50b-42e0-4e42-ac68-0cbb6cd9293b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:55.022212 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b0c7f50b-42e0-4e42-ac68-0cbb6cd9293b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:55.023189 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b0c7f50b-42e0-4e42-ac68-0cbb6cd9293b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:55.023750 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b0c7f50b-42e0-4e42-ac68-0cbb6cd9293b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:55.024274 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b0c7f50b-42e0-4e42-ac68-0cbb6cd9293b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:55.143628 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-1c194e85-f165-4c29-afcc-9c6fb49dda22 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] HTTP exception thrown: Instance 04eedf41-b3f2-4b33-b524-988daff99fdc is not attached to volume 4bfce979-c3b3-4e9d-b857-326102f2c9ca Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1c194e85-f165-4c29-afcc-9c6fb49dda22 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Request method failure captured: Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: DELETE /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments/4bfce979-c3b3-4e9d-b857-326102f2c9ca HTTP/1.1 Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Trww0Y5oEuL1nqjDSgzB1xuiHqVoFqqA3ipFMWCS5WB1klFXWXLaieb9SgOvOlxJGAQ21PZUBDlA5rJifuiIL074PHtjLUct89w57pwYwAt1ZFyXTaLrI8K7B9F1pImo-s5G7d6grDgnCtkk39IwofWU6jGeZtzEnNNGYIQhTKLws Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 6f99f17a6cab46039419eb3b44bdd142 Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-TestMinimumBasicScenario-356703803 Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:49:55.146339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:49:55.148179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/6f99f17a6cab46039419eb3b44bdd142"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6f99f17a6cab46039419eb3b44bdd142"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_6f99f17a6cab46039419eb3b44bdd142"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6f99f17a6cab46039419eb3b44bdd142"}]}] Oct 28 09:49:55.148179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-TestMinimumBasicScenario-356703803 Oct 28 09:49:55.148179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 6f99f17a6cab46039419eb3b44bdd142 Oct 28 09:49:55.148179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-TestMinimumBasicScenario-356703803 Oct 28 09:49:55.148179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-TestMinimumBasicScenario-356703803-project Oct 28 09:49:55.148179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:49:55.148179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:49:55.148179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 23acf07cede847a69ca839ee200accd3 Oct 28 09:49:55.148179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-TestMinimumBasicScenario-356703803-project Oct 28 09:49:55.148179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:49:55.148179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 04eedf41-b3f2-4b33-b524-988daff99fdc is not attached to volume 4bfce979-c3b3-4e9d-b857-326102f2c9ca Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'server_id': '04eedf41-b3f2-4b33-b524-988daff99fdc', 'id': '4bfce979-c3b3-4e9d-b857-326102f2c9ca'} Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 4bfce979-c3b3-4e9d-b857-326102f2c9ca. Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:49:55.149003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 04eedf41-b3f2-4b33-b524-988daff99fdc is not attached to volume 4bfce979-c3b3-4e9d-b857-326102f2c9ca Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 04eedf41-b3f2-4b33-b524-988daff99fdc is not attached to volume 4bfce979-c3b3-4e9d-b857-326102f2c9ca Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:49:55.150299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1c194e85-f165-4c29-afcc-9c6fb49dda22 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Returning 404 to user: Instance 04eedf41-b3f2-4b33-b524-988daff99fdc is not attached to volume 4bfce979-c3b3-4e9d-b857-326102f2c9ca {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:49:55.151398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1c194e85-f165-4c29-afcc-9c6fb49dda22 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "DELETE /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments/4bfce979-c3b3-4e9d-b857-326102f2c9ca" status: 404 len: 154 microversion: 2.1 time: 0.259996 Oct 28 09:49:55.161438 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 342/699] 10.222.0.25 () {58 vars in 1416 bytes} [Fri Oct 28 09:49:54 2022] DELETE /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc/os-volume_attachments/4bfce979-c3b3-4e9d-b857-326102f2c9ca => generated 154 bytes in 264 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:49:55.707932 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-912b35b7-3061-4e3a-9506-7f90217ad030 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:55.708652 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-912b35b7-3061-4e3a-9506-7f90217ad030 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:55.717354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-912b35b7-3061-4e3a-9506-7f90217ad030 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:55.718076 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-912b35b7-3061-4e3a-9506-7f90217ad030 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:55.718852 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-912b35b7-3061-4e3a-9506-7f90217ad030 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:55.719558 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-912b35b7-3061-4e3a-9506-7f90217ad030 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:55.720582 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-912b35b7-3061-4e3a-9506-7f90217ad030 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:55.721005 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-912b35b7-3061-4e3a-9506-7f90217ad030 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:55.755771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-b0c7f50b-42e0-4e42-ac68-0cbb6cd9293b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:55.763827 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b0c7f50b-42e0-4e42-ac68-0cbb6cd9293b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.764529 Oct 28 09:49:55.764326 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 358/700] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:54 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 766 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:55.985385 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-912b35b7-3061-4e3a-9506-7f90217ad030 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:55.994733 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-912b35b7-3061-4e3a-9506-7f90217ad030 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:55.996482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-912b35b7-3061-4e3a-9506-7f90217ad030 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:55.997962 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-912b35b7-3061-4e3a-9506-7f90217ad030 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1723 microversion: 2.20 time: 0.294493 Oct 28 09:49:55.998563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 343/701] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:49:55 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1723 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:49:56.313232 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-95aae637-42a0-4d8b-bb61-118805a337e0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:56.314524 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-95aae637-42a0-4d8b-bb61-118805a337e0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:56.324359 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-95aae637-42a0-4d8b-bb61-118805a337e0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:56.324972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-95aae637-42a0-4d8b-bb61-118805a337e0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:56.325503 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-95aae637-42a0-4d8b-bb61-118805a337e0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:56.326335 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-95aae637-42a0-4d8b-bb61-118805a337e0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:56.326930 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-95aae637-42a0-4d8b-bb61-118805a337e0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:56.327729 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-95aae637-42a0-4d8b-bb61-118805a337e0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:56.361957 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-95aae637-42a0-4d8b-bb61-118805a337e0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Going to try to terminate instance {{(pid=127029) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:49:56.452171 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-95aae637-42a0-4d8b-bb61-118805a337e0 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "DELETE /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 204 len: 0 microversion: 2.1 time: 0.143569 Oct 28 09:49:56.453075 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 359/702] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:49:56 2022] DELETE /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:49:56.461916 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6f77c5ea-2d78-4fea-8aaf-67be7f806a1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:56.462750 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6f77c5ea-2d78-4fea-8aaf-67be7f806a1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:56.470559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f77c5ea-2d78-4fea-8aaf-67be7f806a1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:56.471381 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f77c5ea-2d78-4fea-8aaf-67be7f806a1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:56.472111 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f77c5ea-2d78-4fea-8aaf-67be7f806a1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:56.473145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f77c5ea-2d78-4fea-8aaf-67be7f806a1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:56.473758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f77c5ea-2d78-4fea-8aaf-67be7f806a1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:56.474666 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f77c5ea-2d78-4fea-8aaf-67be7f806a1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:56.567135 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-0b5b7079-dbf8-4848-91bb-f653a05e85be req-a41550ea-0079-4936-a2c3-4e0ca8b3d02b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c716d5fd-f257-4f6a-8f52-18ad59e5c31e", "name": "network-vif-deleted", "tag": "c2310b44-82bb-408f-85b5-8a5556b92e25"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:56.581081 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-0b5b7079-dbf8-4848-91bb-f653a05e85be req-a41550ea-0079-4936-a2c3-4e0ca8b3d02b service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:56.581884 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-0b5b7079-dbf8-4848-91bb-f653a05e85be req-a41550ea-0079-4936-a2c3-4e0ca8b3d02b service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:56.582588 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-0b5b7079-dbf8-4848-91bb-f653a05e85be req-a41550ea-0079-4936-a2c3-4e0ca8b3d02b service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:56.609082 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-0b5b7079-dbf8-4848-91bb-f653a05e85be req-a41550ea-0079-4936-a2c3-4e0ca8b3d02b service nova] Creating event network-vif-deleted:c2310b44-82bb-408f-85b5-8a5556b92e25 for instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e on hv-cinder-zed-3 Oct 28 09:49:56.616341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-0b5b7079-dbf8-4848-91bb-f653a05e85be req-a41550ea-0079-4936-a2c3-4e0ca8b3d02b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054835 Oct 28 09:49:56.620862 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 360/703] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:49:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:56.689774 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6f77c5ea-2d78-4fea-8aaf-67be7f806a1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:56.697860 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6f77c5ea-2d78-4fea-8aaf-67be7f806a1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1554 microversion: 2.1 time: 0.240344 Oct 28 09:49:56.698452 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 344/704] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:56 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1554 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:56.788133 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-a7b81eea-194a-4be8-bbac-1ba37c362b18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:56.788756 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-a7b81eea-194a-4be8-bbac-1ba37c362b18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:56.795406 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a7b81eea-194a-4be8-bbac-1ba37c362b18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:56.795895 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a7b81eea-194a-4be8-bbac-1ba37c362b18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:56.796332 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a7b81eea-194a-4be8-bbac-1ba37c362b18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:56.797037 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a7b81eea-194a-4be8-bbac-1ba37c362b18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:56.797492 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a7b81eea-194a-4be8-bbac-1ba37c362b18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:56.797940 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a7b81eea-194a-4be8-bbac-1ba37c362b18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:56.983786 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-a7b81eea-194a-4be8-bbac-1ba37c362b18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:56.991525 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-a7b81eea-194a-4be8-bbac-1ba37c362b18 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.207623 Oct 28 09:49:56.992101 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 361/705] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:56 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:57.026949 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-eb3d1d45-ac17-4155-870e-0c3165f55849 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:57.027553 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-eb3d1d45-ac17-4155-870e-0c3165f55849 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:57.034732 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eb3d1d45-ac17-4155-870e-0c3165f55849 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:57.035244 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eb3d1d45-ac17-4155-870e-0c3165f55849 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:57.035755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eb3d1d45-ac17-4155-870e-0c3165f55849 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:57.036446 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eb3d1d45-ac17-4155-870e-0c3165f55849 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:57.036887 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eb3d1d45-ac17-4155-870e-0c3165f55849 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:57.037302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eb3d1d45-ac17-4155-870e-0c3165f55849 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:57.102866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-eb3d1d45-ac17-4155-870e-0c3165f55849 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:57.111757 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-eb3d1d45-ac17-4155-870e-0c3165f55849 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:57.113185 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-eb3d1d45-ac17-4155-870e-0c3165f55849 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:57.114491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-eb3d1d45-ac17-4155-870e-0c3165f55849 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1655 microversion: 2.20 time: 0.091547 Oct 28 09:49:57.115050 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 345/706] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:49:57 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1655 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:49:57.722170 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-7a727bba-ed9c-4682-a410-aa26a9695a46 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:57.722835 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-7a727bba-ed9c-4682-a410-aa26a9695a46 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:57.729277 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7a727bba-ed9c-4682-a410-aa26a9695a46 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:57.729757 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7a727bba-ed9c-4682-a410-aa26a9695a46 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:57.730175 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7a727bba-ed9c-4682-a410-aa26a9695a46 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:57.730892 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7a727bba-ed9c-4682-a410-aa26a9695a46 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:57.731344 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7a727bba-ed9c-4682-a410-aa26a9695a46 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:57.731759 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7a727bba-ed9c-4682-a410-aa26a9695a46 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:57.874422 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7a727bba-ed9c-4682-a410-aa26a9695a46 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:57.882354 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-7a727bba-ed9c-4682-a410-aa26a9695a46 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1554 microversion: 2.1 time: 0.164563 Oct 28 09:49:57.883187 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 362/707] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:57 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1554 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:49:57.972010 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-709ba65a-81c4-4b55-a0bf-402500cae5b1 req-c6e1594c-b155-4e20-b5f8-72511d5da372 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "576e3364-9c2e-4360-873a-7a9ded31e8e3", "name": "network-vif-plugged", "status": "completed", "tag": "b0640bfe-73f9-4cda-a3ed-720c31e1391d"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:57.982952 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-709ba65a-81c4-4b55-a0bf-402500cae5b1 req-c6e1594c-b155-4e20-b5f8-72511d5da372 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:57.983438 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-709ba65a-81c4-4b55-a0bf-402500cae5b1 req-c6e1594c-b155-4e20-b5f8-72511d5da372 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:57.983889 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-709ba65a-81c4-4b55-a0bf-402500cae5b1 req-c6e1594c-b155-4e20-b5f8-72511d5da372 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:57.995115 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-709ba65a-81c4-4b55-a0bf-402500cae5b1 req-c6e1594c-b155-4e20-b5f8-72511d5da372 service nova] Creating event network-vif-plugged:b0640bfe-73f9-4cda-a3ed-720c31e1391d for instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 on hv-cinder-zed-3 Oct 28 09:49:58.000455 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-709ba65a-81c4-4b55-a0bf-402500cae5b1 req-c6e1594c-b155-4e20-b5f8-72511d5da372 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032739 Oct 28 09:49:58.001044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 346/708] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:49:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:49:58.015221 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-4fdd05c9-ae55-4395-bb19-47ce3554f953 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:58.015940 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-4fdd05c9-ae55-4395-bb19-47ce3554f953 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:58.022602 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4fdd05c9-ae55-4395-bb19-47ce3554f953 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:58.023430 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4fdd05c9-ae55-4395-bb19-47ce3554f953 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:58.023868 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4fdd05c9-ae55-4395-bb19-47ce3554f953 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:58.024562 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4fdd05c9-ae55-4395-bb19-47ce3554f953 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:58.025025 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4fdd05c9-ae55-4395-bb19-47ce3554f953 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:58.025438 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4fdd05c9-ae55-4395-bb19-47ce3554f953 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:58.142015 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-505c9b8e-69ab-44ef-8f7a-088bb68fef4b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:58.142985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-505c9b8e-69ab-44ef-8f7a-088bb68fef4b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:58.150050 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-505c9b8e-69ab-44ef-8f7a-088bb68fef4b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:58.150561 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-505c9b8e-69ab-44ef-8f7a-088bb68fef4b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:58.151015 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-505c9b8e-69ab-44ef-8f7a-088bb68fef4b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:58.151711 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-505c9b8e-69ab-44ef-8f7a-088bb68fef4b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:58.152150 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-505c9b8e-69ab-44ef-8f7a-088bb68fef4b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:58.152560 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-505c9b8e-69ab-44ef-8f7a-088bb68fef4b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:58.158488 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-4fdd05c9-ae55-4395-bb19-47ce3554f953 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:58.166108 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-4fdd05c9-ae55-4395-bb19-47ce3554f953 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.154757 Oct 28 09:49:58.166704 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 363/709] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:58 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:58.237366 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-505c9b8e-69ab-44ef-8f7a-088bb68fef4b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:58.246214 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-505c9b8e-69ab-44ef-8f7a-088bb68fef4b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:58.247672 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-505c9b8e-69ab-44ef-8f7a-088bb68fef4b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:58.248809 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-505c9b8e-69ab-44ef-8f7a-088bb68fef4b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1655 microversion: 2.20 time: 0.112198 Oct 28 09:49:58.249363 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 347/710] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:49:58 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1655 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:49:58.906234 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-7690172e-b361-49d1-a93c-57a887d9e20d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:58.906874 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-7690172e-b361-49d1-a93c-57a887d9e20d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:58.913438 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7690172e-b361-49d1-a93c-57a887d9e20d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:58.913919 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7690172e-b361-49d1-a93c-57a887d9e20d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:58.914356 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7690172e-b361-49d1-a93c-57a887d9e20d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:58.915074 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7690172e-b361-49d1-a93c-57a887d9e20d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:58.915518 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7690172e-b361-49d1-a93c-57a887d9e20d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:58.916160 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-7690172e-b361-49d1-a93c-57a887d9e20d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:59.149327 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-7690172e-b361-49d1-a93c-57a887d9e20d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:59.158029 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-7690172e-b361-49d1-a93c-57a887d9e20d tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1554 microversion: 2.1 time: 0.255134 Oct 28 09:49:59.158566 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 364/711] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:58 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1554 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:59.189277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-28c869a0-2664-42fa-9894-b151977779e5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:59.189877 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-28c869a0-2664-42fa-9894-b151977779e5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:59.196555 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-28c869a0-2664-42fa-9894-b151977779e5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:59.197024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-28c869a0-2664-42fa-9894-b151977779e5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:59.197446 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-28c869a0-2664-42fa-9894-b151977779e5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:59.198121 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-28c869a0-2664-42fa-9894-b151977779e5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:59.198595 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-28c869a0-2664-42fa-9894-b151977779e5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:59.199006 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-28c869a0-2664-42fa-9894-b151977779e5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:59.279548 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:49:59.280146 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:49:59.287178 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:59.289546 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:59.289546 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:59.289546 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:59.289546 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:59.289892 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:59.326027 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:59.326547 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:59.327018 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:59.327469 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.objects.instance [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lazy-loading 'fault' on Instance uuid c716d5fd-f257-4f6a-8f52-18ad59e5c31e {{(pid=127029) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:49:59.386944 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:59.396784 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:59.398232 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:59.399449 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e0ac68be-3803-4b30-9a87-6bef471ea6be tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1371 microversion: 2.20 time: 0.125007 Oct 28 09:49:59.399997 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 365/712] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:49:59 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1371 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:49:59.447497 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-28c869a0-2664-42fa-9894-b151977779e5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:49:59.458702 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-28c869a0-2664-42fa-9894-b151977779e5 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.273150 Oct 28 09:49:59.460088 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 348/713] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:49:59 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:49:59.498156 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-80a1313a-d55e-4b6e-a272-2c6610aa0d90 req-5f646450-4f24-4ba9-b8b4-f5c7067f089d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "04eedf41-b3f2-4b33-b524-988daff99fdc", "name": "network-vif-deleted", "tag": "0ec06b02-a12d-496d-a40d-c56118810a83"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:49:59.507552 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-80a1313a-d55e-4b6e-a272-2c6610aa0d90 req-5f646450-4f24-4ba9-b8b4-f5c7067f089d service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:49:59.508017 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-80a1313a-d55e-4b6e-a272-2c6610aa0d90 req-5f646450-4f24-4ba9-b8b4-f5c7067f089d service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:49:59.508434 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-80a1313a-d55e-4b6e-a272-2c6610aa0d90 req-5f646450-4f24-4ba9-b8b4-f5c7067f089d service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:49:59.520455 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-80a1313a-d55e-4b6e-a272-2c6610aa0d90 req-5f646450-4f24-4ba9-b8b4-f5c7067f089d service nova] Creating event network-vif-deleted:0ec06b02-a12d-496d-a40d-c56118810a83 for instance 04eedf41-b3f2-4b33-b524-988daff99fdc on hv-cinder-zed-3 Oct 28 09:49:59.525254 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-80a1313a-d55e-4b6e-a272-2c6610aa0d90 req-5f646450-4f24-4ba9-b8b4-f5c7067f089d service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030777 Oct 28 09:49:59.525842 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 366/714] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:49:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:50:00.183338 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-06e45006-b6e7-4c88-bde6-2407933d5d1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:00.184185 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-06e45006-b6e7-4c88-bde6-2407933d5d1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:00.195627 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-06e45006-b6e7-4c88-bde6-2407933d5d1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:00.196626 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-06e45006-b6e7-4c88-bde6-2407933d5d1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:00.197257 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-06e45006-b6e7-4c88-bde6-2407933d5d1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:00.198352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-06e45006-b6e7-4c88-bde6-2407933d5d1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:00.199331 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-06e45006-b6e7-4c88-bde6-2407933d5d1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:00.200377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-06e45006-b6e7-4c88-bde6-2407933d5d1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:00.272813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-06e45006-b6e7-4c88-bde6-2407933d5d1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:00.281493 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-06e45006-b6e7-4c88-bde6-2407933d5d1c tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1343 microversion: 2.1 time: 0.103510 Oct 28 09:50:00.282094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 349/715] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:00 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1343 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:00.431720 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:00.432478 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:00.439726 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:00.440218 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:00.440892 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:00.441619 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:00.442075 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:00.442555 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:00.482655 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:00.483154 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:00.483597 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:00.484057 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.objects.instance [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lazy-loading 'fault' on Instance uuid c716d5fd-f257-4f6a-8f52-18ad59e5c31e {{(pid=127029) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:50:00.491316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-05128f1b-0122-4ac9-89fb-13df8ca5e247 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:00.492435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-05128f1b-0122-4ac9-89fb-13df8ca5e247 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:00.503662 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-05128f1b-0122-4ac9-89fb-13df8ca5e247 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:00.504513 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-05128f1b-0122-4ac9-89fb-13df8ca5e247 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:00.505496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-05128f1b-0122-4ac9-89fb-13df8ca5e247 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:00.506991 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-05128f1b-0122-4ac9-89fb-13df8ca5e247 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:00.508016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-05128f1b-0122-4ac9-89fb-13df8ca5e247 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:00.508972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-05128f1b-0122-4ac9-89fb-13df8ca5e247 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:00.547686 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:00.568637 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:00.571194 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:00.572629 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2c8b1476-ded9-44cd-a03e-4f5b8fba016a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 200 len: 1371 microversion: 2.20 time: 0.150583 Oct 28 09:50:00.573980 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 367/716] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:50:00 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 1371 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:50:00.758535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-05128f1b-0122-4ac9-89fb-13df8ca5e247 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:00.767619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-05128f1b-0122-4ac9-89fb-13df8ca5e247 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1367 microversion: 2.1 time: 0.286200 Oct 28 09:50:00.768214 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 350/717] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:00 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1367 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:01.321250 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:01.322365 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:01.330908 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:01.331886 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:01.332839 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:01.333996 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:01.334998 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:01.335704 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:01.377708 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:01.383176 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:01.383868 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:01.384490 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:01.385317 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.objects.instance [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lazy-loading 'fault' on Instance uuid 04eedf41-b3f2-4b33-b524-988daff99fdc {{(pid=127029) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:50:01.446894 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:01.460748 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-163dd18e-cb0a-4e2a-a322-63863a0a51e2 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1348 microversion: 2.1 time: 0.143470 Oct 28 09:50:01.461570 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 368/718] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:01 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1348 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:50:01.604982 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c5507fe0-64c8-4a5b-8a0f-da25c75753b8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:01.605636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c5507fe0-64c8-4a5b-8a0f-da25c75753b8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:01.612924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c5507fe0-64c8-4a5b-8a0f-da25c75753b8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:01.613419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c5507fe0-64c8-4a5b-8a0f-da25c75753b8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:01.613856 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c5507fe0-64c8-4a5b-8a0f-da25c75753b8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:01.614602 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c5507fe0-64c8-4a5b-8a0f-da25c75753b8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:01.615062 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c5507fe0-64c8-4a5b-8a0f-da25c75753b8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:01.615480 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c5507fe0-64c8-4a5b-8a0f-da25c75753b8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:01.630160 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-c5507fe0-64c8-4a5b-8a0f-da25c75753b8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] HTTP exception thrown: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c5507fe0-64c8-4a5b-8a0f-da25c75753b8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Request method failure captured: Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e HTTP/1.1 Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Skf5aWDdC40CR74ytCMLUExvGa9v-WnWSZgir8FnsqOIPgcQbm8rt7LfqPr3K-OpviTVPDVrFgjoVWp5XTqu5zvyFdSr2QLXhFcNRteklZzYEfWKkCO2Kw-We7GIm7OBNlI_jJkjuNbTKEij2gaBf7wIZ1vIvR2dikCsL6NQ3ueTQ Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Openstack-Nova-Api-Version: 2.20 Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 29c47b13a2044939b5446ffe5397633a Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:50:01.631873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:50:01.634419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/29c47b13a2044939b5446ffe5397633a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/29c47b13a2044939b5446ffe5397633a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_29c47b13a2044939b5446ffe5397633a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/29c47b13a2044939b5446ffe5397633a"}]}] Oct 28 09:50:01.634419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:50:01.634419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 29c47b13a2044939b5446ffe5397633a Oct 28 09:50:01.634419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:50:01.634419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeShelveTestJSON-387501597-project Oct 28 09:50:01.634419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:50:01.634419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:50:01.634419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 715a782e5a56431e88e4fe5a3b0a62aa Oct 28 09:50:01.634419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeShelveTestJSON-387501597-project Oct 28 09:50:01.634419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:50:01.634419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:50:01.634419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'c716d5fd-f257-4f6a-8f52-18ad59e5c31e'} Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:50:01.635843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:50:01.638190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:01.640223 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c5507fe0-64c8-4a5b-8a0f-da25c75753b8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Returning 404 to user: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:50:01.640223 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c5507fe0-64c8-4a5b-8a0f-da25c75753b8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 404 len: 111 microversion: 2.20 time: 0.032237 Oct 28 09:50:01.640223 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 351/719] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:50:01 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 28 09:50:01.646610 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-df74f130-32f5-4959-9860-cd74cdf4f66c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '.version_select at 0x7f271a282dc0>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:01.678002 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-df74f130-32f5-4959-9860-cd74cdf4f66c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1471611763" status: 204 len: 0 microversion: 2.20 time: 0.038618 Oct 28 09:50:01.678616 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 369/720] 10.222.0.25 () {60 vars in 1262 bytes} [Fri Oct 28 09:50:01 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1471611763 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 28 09:50:01.791556 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-dad55787-c208-4c38-a960-8e95fa663de7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:01.792173 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-dad55787-c208-4c38-a960-8e95fa663de7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:01.799062 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dad55787-c208-4c38-a960-8e95fa663de7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:01.799552 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dad55787-c208-4c38-a960-8e95fa663de7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:01.799987 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dad55787-c208-4c38-a960-8e95fa663de7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:01.800684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dad55787-c208-4c38-a960-8e95fa663de7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:01.801131 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dad55787-c208-4c38-a960-8e95fa663de7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:01.801551 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dad55787-c208-4c38-a960-8e95fa663de7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:02.153053 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-dad55787-c208-4c38-a960-8e95fa663de7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:02.166677 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-dad55787-c208-4c38-a960-8e95fa663de7 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1558 microversion: 2.1 time: 0.378612 Oct 28 09:50:02.167626 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 352/721] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:01 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1558 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:02.195642 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c1f16ad8-84c5-4a7f-ab6f-6f420dc8a72b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:02.196522 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c1f16ad8-84c5-4a7f-ab6f-6f420dc8a72b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:02.211198 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c1f16ad8-84c5-4a7f-ab6f-6f420dc8a72b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:02.212049 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c1f16ad8-84c5-4a7f-ab6f-6f420dc8a72b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:02.212591 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c1f16ad8-84c5-4a7f-ab6f-6f420dc8a72b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:02.213426 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c1f16ad8-84c5-4a7f-ab6f-6f420dc8a72b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:02.214002 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c1f16ad8-84c5-4a7f-ab6f-6f420dc8a72b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:02.214732 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c1f16ad8-84c5-4a7f-ab6f-6f420dc8a72b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:02.222482 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:02.384485 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c1f16ad8-84c5-4a7f-ab6f-6f420dc8a72b tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.196560 Oct 28 09:50:02.386102 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 370/722] 10.222.0.25 () {58 vars in 1275 bytes} [Fri Oct 28 09:50:02 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/os-interface => generated 285 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:50:02.492187 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:02.493698 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:02.504837 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:02.505752 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:02.506475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:02.507987 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:02.508817 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:02.509588 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:02.522041 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=100 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:02.558721 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:02.559522 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:02.560179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:02.561548 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lazy-loading 'fault' on Instance uuid 04eedf41-b3f2-4b33-b524-988daff99fdc {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:50:02.655569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '23acf07cede847a69ca839ee200accd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f99f17a6cab46039419eb3b44bdd142', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:02.666056 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5fe90feb-d989-4c80-8533-e75bc1719309 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 200 len: 1348 microversion: 2.1 time: 0.181248 Oct 28 09:50:02.667316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 353/723] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:02 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 1348 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:50:02.891009 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-4257d915-a17d-4033-b878-b476e8b997bb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Action: 'create', calling method: .version_select at 0x7f271a1d09d0>, body: {"keypair": {"name": "tempest-keypair-853652994", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNAE+RD8mbnQ4DyA0UFZYuDI/lU3tHnmQXO1uJCWSMeQEam4zpLHdv+mDyKPv865BS6C6288tQbmzp1/WHwnXn3YNpyM46CDUW7vqZD6AaPFrpJrt/H1tpSxK7Kah7Zg1A=="}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:02.915236 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-4257d915-a17d-4033-b878-b476e8b997bb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Getting quotas for project 29c47b13a2044939b5446ffe5397633a. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:50:02.922649 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-4257d915-a17d-4033-b878-b476e8b997bb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Getting quotas for user 715a782e5a56431e88e4fe5a3b0a62aa and project 29c47b13a2044939b5446ffe5397633a. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:50:02.958897 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-4257d915-a17d-4033-b878-b476e8b997bb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.074826 Oct 28 09:50:02.959512 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 371/724] 10.222.0.25 () {62 vars in 1200 bytes} [Fri Oct 28 09:50:02 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 76 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Oct 28 09:50:03.709808 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5ec9056f-ad72-47c8-97c6-a25d4e8b7629 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:03.710765 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5ec9056f-ad72-47c8-97c6-a25d4e8b7629 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] [instance: 04eedf41-b3f2-4b33-b524-988daff99fdc] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:03.718962 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5ec9056f-ad72-47c8-97c6-a25d4e8b7629 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:03.720239 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5ec9056f-ad72-47c8-97c6-a25d4e8b7629 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:03.720773 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5ec9056f-ad72-47c8-97c6-a25d4e8b7629 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:03.721704 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5ec9056f-ad72-47c8-97c6-a25d4e8b7629 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:03.722508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5ec9056f-ad72-47c8-97c6-a25d4e8b7629 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:03.723145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5ec9056f-ad72-47c8-97c6-a25d4e8b7629 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:03.742099 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-5ec9056f-ad72-47c8-97c6-a25d4e8b7629 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] HTTP exception thrown: Instance 04eedf41-b3f2-4b33-b524-988daff99fdc could not be found. Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5ec9056f-ad72-47c8-97c6-a25d4e8b7629 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Request method failure captured: Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc HTTP/1.1 Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Trww0Y5oEuL1nqjDSgzB1xuiHqVoFqqA3ipFMWCS5WB1klFXWXLaieb9SgOvOlxJGAQ21PZUBDlA5rJifuiIL074PHtjLUct89w57pwYwAt1ZFyXTaLrI8K7B9F1pImo-s5G7d6grDgnCtkk39IwofWU6jGeZtzEnNNGYIQhTKLws Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 6f99f17a6cab46039419eb3b44bdd142 Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-TestMinimumBasicScenario-356703803 Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:50:03.744249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:50:03.746909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/6f99f17a6cab46039419eb3b44bdd142"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6f99f17a6cab46039419eb3b44bdd142"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_6f99f17a6cab46039419eb3b44bdd142"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6f99f17a6cab46039419eb3b44bdd142"}]}] Oct 28 09:50:03.746909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-TestMinimumBasicScenario-356703803 Oct 28 09:50:03.746909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 6f99f17a6cab46039419eb3b44bdd142 Oct 28 09:50:03.746909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-TestMinimumBasicScenario-356703803 Oct 28 09:50:03.746909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-TestMinimumBasicScenario-356703803-project Oct 28 09:50:03.746909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:50:03.746909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:50:03.746909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 23acf07cede847a69ca839ee200accd3 Oct 28 09:50:03.746909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-TestMinimumBasicScenario-356703803-project Oct 28 09:50:03.746909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:50:03.746909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 04eedf41-b3f2-4b33-b524-988daff99fdc could not be found. Oct 28 09:50:03.746909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '04eedf41-b3f2-4b33-b524-988daff99fdc'} Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 04eedf41-b3f2-4b33-b524-988daff99fdc could not be found. Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:50:03.748456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 04eedf41-b3f2-4b33-b524-988daff99fdc could not be found. Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 04eedf41-b3f2-4b33-b524-988daff99fdc could not be found. Oct 28 09:50:03.750954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:03.754504 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5ec9056f-ad72-47c8-97c6-a25d4e8b7629 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Returning 404 to user: Instance 04eedf41-b3f2-4b33-b524-988daff99fdc could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:50:03.755944 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5ec9056f-ad72-47c8-97c6-a25d4e8b7629 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc" status: 404 len: 111 microversion: 2.1 time: 0.053029 Oct 28 09:50:03.760432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 354/725] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:03 2022] GET /compute/v2.1/servers/04eedf41-b3f2-4b33-b524-988daff99fdc => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:50:03.769180 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d193b34b-df2d-47cd-b06c-07087c87c536 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] Calling method '.version_select at 0x7f271a206940>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:03.806790 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d193b34b-df2d-47cd-b06c-07087c87c536 tempest-TestMinimumBasicScenario-356703803 tempest-TestMinimumBasicScenario-356703803-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-498428939" status: 202 len: 0 microversion: 2.1 time: 0.042917 Oct 28 09:50:03.811322 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 372/726] 10.222.0.25 () {58 vars in 1269 bytes} [Fri Oct 28 09:50:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-498428939 => generated 0 bytes in 50 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:50:04.444745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-b8771c1b-3c8a-4d4d-a4e6-49ea00feb66b req-129e9299-dc41-47c3-9f9d-6e4cd6863142 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "576e3364-9c2e-4360-873a-7a9ded31e8e3", "tag": "b0640bfe-73f9-4cda-a3ed-720c31e1391d"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:04.477846 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-b8771c1b-3c8a-4d4d-a4e6-49ea00feb66b req-129e9299-dc41-47c3-9f9d-6e4cd6863142 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:04.479365 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-b8771c1b-3c8a-4d4d-a4e6-49ea00feb66b req-129e9299-dc41-47c3-9f9d-6e4cd6863142 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:04.480681 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-b8771c1b-3c8a-4d4d-a4e6-49ea00feb66b req-129e9299-dc41-47c3-9f9d-6e4cd6863142 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:04.520758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-b8771c1b-3c8a-4d4d-a4e6-49ea00feb66b req-129e9299-dc41-47c3-9f9d-6e4cd6863142 service nova] Creating event network-changed:b0640bfe-73f9-4cda-a3ed-720c31e1391d for instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 on hv-cinder-zed-3 Oct 28 09:50:04.534309 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-b8771c1b-3c8a-4d4d-a4e6-49ea00feb66b req-129e9299-dc41-47c3-9f9d-6e4cd6863142 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.112686 Oct 28 09:50:04.537210 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 355/727] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:50:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:50:05.513846 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1374745559", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "1c06056f-c33e-47c9-b078-4c3c3f047c82"}], "security_groups": [{"name": "tempest-securitygroup--23944376"}], "key_name": "tempest-keypair-853652994", "user_data": "}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:05.838180 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.network.neutron [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] validate_networks() for [('1c06056f-c33e-47c9-b078-4c3c3f047c82', None, None, None, None, None)] {{(pid=127029) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:50:07.439473 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Flavor limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:50:07.440133 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Image limits 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:50:07.440590 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Flavor pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:50:07.441072 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Image pref 0:0:0 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:50:07.441559 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.virt.hardware [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127029) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:50:07.451074 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:07.451556 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:07.451993 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:07.452721 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:07.453177 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:07.453627 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:07.472453 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Getting quotas for project 29c47b13a2044939b5446ffe5397633a. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:50:07.477081 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Getting quotas for user 715a782e5a56431e88e4fe5a3b0a62aa and project 29c47b13a2044939b5446ffe5397633a. Resources: {'instances', 'ram', 'cores'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:50:07.489834 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Going to run 1 instances... {{(pid=127029) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:50:07.492300 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127029) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:50:07.556521 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2765bbce-120e-4cff-b050-cb54a9ab4f21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.20 time: 2.047530 Oct 28 09:50:07.557974 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 373/728] 10.222.0.25 () {62 vars in 1188 bytes} [Fri Oct 28 09:50:05 2022] POST /compute/v2.1/servers => generated 400 bytes in 2050 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Oct 28 09:50:07.576329 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-493c7d5d-f62e-408a-8fc1-dac68754af40 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:07.577614 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-493c7d5d-f62e-408a-8fc1-dac68754af40 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:07.662673 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-493c7d5d-f62e-408a-8fc1-dac68754af40 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:07.673532 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-493c7d5d-f62e-408a-8fc1-dac68754af40 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:07.675397 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-493c7d5d-f62e-408a-8fc1-dac68754af40 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:07.676732 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-493c7d5d-f62e-408a-8fc1-dac68754af40 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1281 microversion: 2.20 time: 0.109719 Oct 28 09:50:07.677284 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 356/729] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:50:07 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1281 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:50:09.065859 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-762234e0-1f63-452a-b644-459a38aa2fcd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:09.068220 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-762234e0-1f63-452a-b644-459a38aa2fcd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:09.081134 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-762234e0-1f63-452a-b644-459a38aa2fcd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:09.082571 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-762234e0-1f63-452a-b644-459a38aa2fcd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:09.083688 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-762234e0-1f63-452a-b644-459a38aa2fcd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:09.085317 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-762234e0-1f63-452a-b644-459a38aa2fcd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:09.086227 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-762234e0-1f63-452a-b644-459a38aa2fcd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:09.087625 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-762234e0-1f63-452a-b644-459a38aa2fcd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:09.210492 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-762234e0-1f63-452a-b644-459a38aa2fcd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:09.228515 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-762234e0-1f63-452a-b644-459a38aa2fcd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:09.230157 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-762234e0-1f63-452a-b644-459a38aa2fcd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:09.231401 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-762234e0-1f63-452a-b644-459a38aa2fcd tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1333 microversion: 2.20 time: 0.532753 Oct 28 09:50:09.231971 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 374/730] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:50:08 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1333 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:50:10.289411 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-69ff0da5-4881-4744-abff-605e92142650 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:10.291734 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-69ff0da5-4881-4744-abff-605e92142650 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:10.306056 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69ff0da5-4881-4744-abff-605e92142650 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:10.306876 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69ff0da5-4881-4744-abff-605e92142650 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:10.307462 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69ff0da5-4881-4744-abff-605e92142650 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:10.308502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69ff0da5-4881-4744-abff-605e92142650 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:10.309463 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69ff0da5-4881-4744-abff-605e92142650 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:10.310101 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69ff0da5-4881-4744-abff-605e92142650 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:10.473824 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-69ff0da5-4881-4744-abff-605e92142650 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:10.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-69ff0da5-4881-4744-abff-605e92142650 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:10.495998 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-69ff0da5-4881-4744-abff-605e92142650 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:10.498228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-69ff0da5-4881-4744-abff-605e92142650 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1339 microversion: 2.20 time: 0.220736 Oct 28 09:50:10.499213 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 357/731] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:50:10 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1339 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:50:11.528446 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-4384d340-b623-4093-b762-ea7b3692e112 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:11.529505 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-4384d340-b623-4093-b762-ea7b3692e112 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:11.545034 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4384d340-b623-4093-b762-ea7b3692e112 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:11.545818 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4384d340-b623-4093-b762-ea7b3692e112 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:11.546669 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4384d340-b623-4093-b762-ea7b3692e112 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:11.548021 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4384d340-b623-4093-b762-ea7b3692e112 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:11.548797 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4384d340-b623-4093-b762-ea7b3692e112 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:11.549569 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-4384d340-b623-4093-b762-ea7b3692e112 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:12.146748 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-4384d340-b623-4093-b762-ea7b3692e112 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:12.156225 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-4384d340-b623-4093-b762-ea7b3692e112 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:12.158990 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-4384d340-b623-4093-b762-ea7b3692e112 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:12.160863 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-4384d340-b623-4093-b762-ea7b3692e112 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1405 microversion: 2.20 time: 0.638693 Oct 28 09:50:12.161721 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 375/732] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:50:11 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1405 bytes in 640 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:50:12.633199 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-51625118-a6a0-4111-8f8f-13198e641dbb req-a5164da9-7f95-4a3e-b7d2-c119d9ba2b66 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b68e375d-f7ab-4d35-a8c7-081596d0e6e2", "tag": "533de8d8-0438-4dd2-946d-f12312377670"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:12.646188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 28 09:50:12.646670 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Fri Oct 28 09:50:12 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.25) !!! Oct 28 09:50:12.660377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-51625118-a6a0-4111-8f8f-13198e641dbb req-a5164da9-7f95-4a3e-b7d2-c119d9ba2b66 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:12.660974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-51625118-a6a0-4111-8f8f-13198e641dbb req-a5164da9-7f95-4a3e-b7d2-c119d9ba2b66 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:12.661599 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-51625118-a6a0-4111-8f8f-13198e641dbb req-a5164da9-7f95-4a3e-b7d2-c119d9ba2b66 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:12.675915 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-51625118-a6a0-4111-8f8f-13198e641dbb req-a5164da9-7f95-4a3e-b7d2-c119d9ba2b66 service nova] Creating event network-changed:533de8d8-0438-4dd2-946d-f12312377670 for instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 on hv-cinder-zed-3 Oct 28 09:50:12.682770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-51625118-a6a0-4111-8f8f-13198e641dbb req-a5164da9-7f95-4a3e-b7d2-c119d9ba2b66 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054108 Oct 28 09:50:12.683857 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 358/733] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:50:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:50:13.189027 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-37819eff-cef8-4beb-8fba-106099167392 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:13.190830 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-37819eff-cef8-4beb-8fba-106099167392 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:13.198569 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-37819eff-cef8-4beb-8fba-106099167392 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:13.199308 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-37819eff-cef8-4beb-8fba-106099167392 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:13.199855 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-37819eff-cef8-4beb-8fba-106099167392 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:13.200972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-37819eff-cef8-4beb-8fba-106099167392 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:13.201689 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-37819eff-cef8-4beb-8fba-106099167392 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:13.202162 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-37819eff-cef8-4beb-8fba-106099167392 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:13.521676 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-37819eff-cef8-4beb-8fba-106099167392 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:13.536463 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-37819eff-cef8-4beb-8fba-106099167392 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:13.539634 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-37819eff-cef8-4beb-8fba-106099167392 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:13.541881 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-37819eff-cef8-4beb-8fba-106099167392 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1405 microversion: 2.20 time: 0.357543 Oct 28 09:50:13.554712 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 376/734] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:50:13 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1405 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:50:14.059110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7a2438bb-d97a-4ce2-8fa1-9e1b1552568b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:14.061506 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7a2438bb-d97a-4ce2-8fa1-9e1b1552568b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:14.069022 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a2438bb-d97a-4ce2-8fa1-9e1b1552568b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:14.069514 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a2438bb-d97a-4ce2-8fa1-9e1b1552568b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:14.069953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a2438bb-d97a-4ce2-8fa1-9e1b1552568b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:14.070720 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a2438bb-d97a-4ce2-8fa1-9e1b1552568b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:14.071158 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a2438bb-d97a-4ce2-8fa1-9e1b1552568b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:14.071575 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a2438bb-d97a-4ce2-8fa1-9e1b1552568b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:14.111477 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7a2438bb-d97a-4ce2-8fa1-9e1b1552568b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.055603 Oct 28 09:50:14.112665 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 359/735] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:50:14 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d/os-volume_attachments => generated 25 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 09:50:14.214339 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8682aae3-3634-4b49-a295-0062cff3c220 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:14.215447 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8682aae3-3634-4b49-a295-0062cff3c220 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:14.228630 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8682aae3-3634-4b49-a295-0062cff3c220 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:14.229359 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8682aae3-3634-4b49-a295-0062cff3c220 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:14.230157 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8682aae3-3634-4b49-a295-0062cff3c220 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:14.231478 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8682aae3-3634-4b49-a295-0062cff3c220 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:14.232313 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8682aae3-3634-4b49-a295-0062cff3c220 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:14.233064 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8682aae3-3634-4b49-a295-0062cff3c220 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:14.285277 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8682aae3-3634-4b49-a295-0062cff3c220 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Going to try to terminate instance {{(pid=127029) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:50:14.428882 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8682aae3-3634-4b49-a295-0062cff3c220 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "DELETE /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 204 len: 0 microversion: 2.1 time: 0.221588 Oct 28 09:50:14.429754 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 377/736] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:50:14 2022] DELETE /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:50:14.441540 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6d847062-cad9-4c36-a1d6-ce417310f9a9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:14.443126 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6d847062-cad9-4c36-a1d6-ce417310f9a9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:14.454814 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d847062-cad9-4c36-a1d6-ce417310f9a9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:14.456030 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d847062-cad9-4c36-a1d6-ce417310f9a9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:14.457292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d847062-cad9-4c36-a1d6-ce417310f9a9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:14.458934 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d847062-cad9-4c36-a1d6-ce417310f9a9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:14.459968 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d847062-cad9-4c36-a1d6-ce417310f9a9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:14.460742 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d847062-cad9-4c36-a1d6-ce417310f9a9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:14.603381 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-e83bab99-2b91-403a-a679-83b00112e6e4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:14.610032 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-e83bab99-2b91-403a-a679-83b00112e6e4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:14.632303 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e83bab99-2b91-403a-a679-83b00112e6e4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:14.634609 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e83bab99-2b91-403a-a679-83b00112e6e4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:14.636378 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e83bab99-2b91-403a-a679-83b00112e6e4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:14.638650 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e83bab99-2b91-403a-a679-83b00112e6e4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:14.639884 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e83bab99-2b91-403a-a679-83b00112e6e4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:14.641027 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-e83bab99-2b91-403a-a679-83b00112e6e4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:14.846050 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6d847062-cad9-4c36-a1d6-ce417310f9a9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:14.867946 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6d847062-cad9-4c36-a1d6-ce417310f9a9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1673 microversion: 2.1 time: 0.432955 Oct 28 09:50:14.869815 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 360/737] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:14 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1673 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:14.916424 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e83bab99-2b91-403a-a679-83b00112e6e4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:14.926140 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e83bab99-2b91-403a-a679-83b00112e6e4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:14.927664 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-e83bab99-2b91-403a-a679-83b00112e6e4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:14.928849 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-e83bab99-2b91-403a-a679-83b00112e6e4 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1405 microversion: 2.20 time: 0.347987 Oct 28 09:50:14.929432 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 378/738] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:50:14 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1405 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:50:15.913807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f6a980fb-901d-4cde-a7a9-f6e507d776d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:15.915000 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f6a980fb-901d-4cde-a7a9-f6e507d776d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:15.932374 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6a980fb-901d-4cde-a7a9-f6e507d776d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:15.933554 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6a980fb-901d-4cde-a7a9-f6e507d776d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:15.934808 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6a980fb-901d-4cde-a7a9-f6e507d776d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:15.936292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6a980fb-901d-4cde-a7a9-f6e507d776d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:15.937026 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6a980fb-901d-4cde-a7a9-f6e507d776d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:15.937910 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6a980fb-901d-4cde-a7a9-f6e507d776d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:15.962218 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c60ef735-f15e-4c66-86c1-385555c11938 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:15.963531 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-c60ef735-f15e-4c66-86c1-385555c11938 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:15.987071 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c60ef735-f15e-4c66-86c1-385555c11938 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:15.987837 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c60ef735-f15e-4c66-86c1-385555c11938 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:15.995112 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c60ef735-f15e-4c66-86c1-385555c11938 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:15.996548 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c60ef735-f15e-4c66-86c1-385555c11938 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:15.997807 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c60ef735-f15e-4c66-86c1-385555c11938 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:16.000501 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-c60ef735-f15e-4c66-86c1-385555c11938 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:16.507379 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f6a980fb-901d-4cde-a7a9-f6e507d776d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:16.515758 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c60ef735-f15e-4c66-86c1-385555c11938 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:16.534458 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f6a980fb-901d-4cde-a7a9-f6e507d776d6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1673 microversion: 2.1 time: 0.628391 Oct 28 09:50:16.535287 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c60ef735-f15e-4c66-86c1-385555c11938 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:16.538633 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 361/739] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:15 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1673 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:16.539202 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-c60ef735-f15e-4c66-86c1-385555c11938 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:16.541417 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-c60ef735-f15e-4c66-86c1-385555c11938 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1405 microversion: 2.20 time: 0.589435 Oct 28 09:50:16.542495 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 379/740] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:50:15 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1405 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:50:17.575177 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b5163b89-f1c5-487c-929d-0d28769ddcf9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:17.575985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b5163b89-f1c5-487c-929d-0d28769ddcf9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:17.584892 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-85fb5c02-de79-4233-abbf-e90e841e9f67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:17.585771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-85fb5c02-de79-4233-abbf-e90e841e9f67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:17.588783 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=2 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:17.589705 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=10 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:17.595310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=68 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:17.599691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b5163b89-f1c5-487c-929d-0d28769ddcf9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:17.600230 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b5163b89-f1c5-487c-929d-0d28769ddcf9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:17.600787 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b5163b89-f1c5-487c-929d-0d28769ddcf9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:17.601511 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b5163b89-f1c5-487c-929d-0d28769ddcf9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:17.601964 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b5163b89-f1c5-487c-929d-0d28769ddcf9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:17.602388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b5163b89-f1c5-487c-929d-0d28769ddcf9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:17.607405 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-85fb5c02-de79-4233-abbf-e90e841e9f67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:17.607951 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-85fb5c02-de79-4233-abbf-e90e841e9f67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:17.608497 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-85fb5c02-de79-4233-abbf-e90e841e9f67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:17.609475 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-85fb5c02-de79-4233-abbf-e90e841e9f67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:17.610088 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-85fb5c02-de79-4233-abbf-e90e841e9f67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:17.610770 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-85fb5c02-de79-4233-abbf-e90e841e9f67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:17.667170 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:18.441223 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b5163b89-f1c5-487c-929d-0d28769ddcf9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:18.464669 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-85fb5c02-de79-4233-abbf-e90e841e9f67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:18.471795 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b5163b89-f1c5-487c-929d-0d28769ddcf9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1673 microversion: 2.1 time: 0.901232 Oct 28 09:50:18.482452 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 362/741] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:17 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1673 bytes in 904 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:18.482675 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-85fb5c02-de79-4233-abbf-e90e841e9f67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:18.482675 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-85fb5c02-de79-4233-abbf-e90e841e9f67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:18.482675 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-85fb5c02-de79-4233-abbf-e90e841e9f67 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1405 microversion: 2.20 time: 0.899413 Oct 28 09:50:18.482980 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 380/742] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:50:17 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1405 bytes in 901 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:50:18.483083 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-329db7ff-e296-4e5b-a0b8-8c61c50761c2 req-7d86be83-a9ea-4fb5-88d3-3cecfe9ac205 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b68e375d-f7ab-4d35-a8c7-081596d0e6e2", "name": "network-vif-plugged", "status": "completed", "tag": "533de8d8-0438-4dd2-946d-f12312377670"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:18.504242 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-329db7ff-e296-4e5b-a0b8-8c61c50761c2 req-7d86be83-a9ea-4fb5-88d3-3cecfe9ac205 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:18.504995 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-329db7ff-e296-4e5b-a0b8-8c61c50761c2 req-7d86be83-a9ea-4fb5-88d3-3cecfe9ac205 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:18.505609 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-329db7ff-e296-4e5b-a0b8-8c61c50761c2 req-7d86be83-a9ea-4fb5-88d3-3cecfe9ac205 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:18.522098 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-329db7ff-e296-4e5b-a0b8-8c61c50761c2 req-7d86be83-a9ea-4fb5-88d3-3cecfe9ac205 service nova] Creating event network-vif-plugged:533de8d8-0438-4dd2-946d-f12312377670 for instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 on hv-cinder-zed-3 Oct 28 09:50:18.527677 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-329db7ff-e296-4e5b-a0b8-8c61c50761c2 req-7d86be83-a9ea-4fb5-88d3-3cecfe9ac205 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052410 Oct 28 09:50:18.528588 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 363/743] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:50:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:50:19.511931 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8ae959ce-cf9a-4252-acfe-79108e93e30d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:19.513740 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8ae959ce-cf9a-4252-acfe-79108e93e30d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:19.524338 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f5290ae1-0f32-4065-95ec-a8311b56ad8a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:19.525197 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f5290ae1-0f32-4065-95ec-a8311b56ad8a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:19.535351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f5290ae1-0f32-4065-95ec-a8311b56ad8a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:19.536277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f5290ae1-0f32-4065-95ec-a8311b56ad8a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:19.536875 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f5290ae1-0f32-4065-95ec-a8311b56ad8a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:19.538516 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f5290ae1-0f32-4065-95ec-a8311b56ad8a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:19.539116 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8ae959ce-cf9a-4252-acfe-79108e93e30d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:19.539982 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f5290ae1-0f32-4065-95ec-a8311b56ad8a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:19.540634 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f5290ae1-0f32-4065-95ec-a8311b56ad8a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:19.540756 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8ae959ce-cf9a-4252-acfe-79108e93e30d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:19.541734 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8ae959ce-cf9a-4252-acfe-79108e93e30d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:19.543380 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8ae959ce-cf9a-4252-acfe-79108e93e30d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:19.544347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8ae959ce-cf9a-4252-acfe-79108e93e30d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:19.545499 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8ae959ce-cf9a-4252-acfe-79108e93e30d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:19.697231 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-8ae959ce-cf9a-4252-acfe-79108e93e30d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:19.710962 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8ae959ce-cf9a-4252-acfe-79108e93e30d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1606 microversion: 2.1 time: 0.205600 Oct 28 09:50:19.727084 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 381/744] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:19 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1606 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:19.929163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f5290ae1-0f32-4065-95ec-a8311b56ad8a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:19.952393 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f5290ae1-0f32-4065-95ec-a8311b56ad8a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:19.970753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f5290ae1-0f32-4065-95ec-a8311b56ad8a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:19.974446 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f5290ae1-0f32-4065-95ec-a8311b56ad8a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1405 microversion: 2.20 time: 0.455811 Oct 28 09:50:19.975432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 364/745] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:50:19 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1405 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:50:20.035136 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-e95e5840-42b6-49b9-8768-d281f4a667da req-cd4a17d3-f373-492d-8c01-3079518f697a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b981eca6-4d6f-4e48-bf62-016833f2885d", "name": "network-vif-deleted", "tag": "e1425374-6966-45a1-bf96-5df1dadc813c"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:20.052657 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-e95e5840-42b6-49b9-8768-d281f4a667da req-cd4a17d3-f373-492d-8c01-3079518f697a service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:20.053571 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-e95e5840-42b6-49b9-8768-d281f4a667da req-cd4a17d3-f373-492d-8c01-3079518f697a service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:20.054576 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-e95e5840-42b6-49b9-8768-d281f4a667da req-cd4a17d3-f373-492d-8c01-3079518f697a service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:20.075454 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-e95e5840-42b6-49b9-8768-d281f4a667da req-cd4a17d3-f373-492d-8c01-3079518f697a service nova] Creating event network-vif-deleted:e1425374-6966-45a1-bf96-5df1dadc813c for instance b981eca6-4d6f-4e48-bf62-016833f2885d on hv-cinder-zed-3 Oct 28 09:50:20.089317 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-e95e5840-42b6-49b9-8768-d281f4a667da req-cd4a17d3-f373-492d-8c01-3079518f697a service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058043 Oct 28 09:50:20.093322 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 382/746] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:50:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:50:20.743453 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:20.744306 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:20.753712 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:20.754473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:20.755383 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:20.756293 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:20.757002 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:20.757733 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:20.804194 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:20.804878 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:20.805404 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:20.806024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lazy-loading 'fault' on Instance uuid b981eca6-4d6f-4e48-bf62-016833f2885d {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:50:20.942932 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:20.959581 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f1c3e957-7c82-415e-b299-ea1c7279627a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 200 len: 1328 microversion: 2.1 time: 0.221644 Oct 28 09:50:20.962560 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 365/747] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:20 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 1328 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:21.028819 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-0d560969-e981-4e6f-83dd-559b8c00f5ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:21.029972 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-0d560969-e981-4e6f-83dd-559b8c00f5ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:21.042134 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d560969-e981-4e6f-83dd-559b8c00f5ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:21.043025 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d560969-e981-4e6f-83dd-559b8c00f5ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:21.044036 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d560969-e981-4e6f-83dd-559b8c00f5ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:21.045559 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d560969-e981-4e6f-83dd-559b8c00f5ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:21.046688 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d560969-e981-4e6f-83dd-559b8c00f5ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:21.047530 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-0d560969-e981-4e6f-83dd-559b8c00f5ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:21.427131 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0d560969-e981-4e6f-83dd-559b8c00f5ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:21.445181 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0d560969-e981-4e6f-83dd-559b8c00f5ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:21.449233 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-0d560969-e981-4e6f-83dd-559b8c00f5ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:21.452043 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-0d560969-e981-4e6f-83dd-559b8c00f5ba tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1597 microversion: 2.20 time: 0.432621 Oct 28 09:50:21.453455 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 383/748] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:50:21 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1597 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:50:21.505658 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9fb76f10-b177-4b20-aaba-9832d8300e58 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:21.506574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9fb76f10-b177-4b20-aaba-9832d8300e58 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:21.518743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9fb76f10-b177-4b20-aaba-9832d8300e58 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:21.519626 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9fb76f10-b177-4b20-aaba-9832d8300e58 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:21.520655 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9fb76f10-b177-4b20-aaba-9832d8300e58 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:21.522269 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9fb76f10-b177-4b20-aaba-9832d8300e58 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:21.523168 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9fb76f10-b177-4b20-aaba-9832d8300e58 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:21.524060 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9fb76f10-b177-4b20-aaba-9832d8300e58 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:21.657753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9fb76f10-b177-4b20-aaba-9832d8300e58 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.159508 Oct 28 09:50:21.658869 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 366/749] 10.222.0.25 () {60 vars in 1316 bytes} [Fri Oct 28 09:50:21 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/os-interface => generated 285 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:50:21.998442 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-94731191-4b92-47fb-a184-ab25161d23f8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:21.999595 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-94731191-4b92-47fb-a184-ab25161d23f8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:22.041223 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94731191-4b92-47fb-a184-ab25161d23f8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:22.044412 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94731191-4b92-47fb-a184-ab25161d23f8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:22.049753 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94731191-4b92-47fb-a184-ab25161d23f8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:22.054678 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94731191-4b92-47fb-a184-ab25161d23f8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:22.059200 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94731191-4b92-47fb-a184-ab25161d23f8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:22.062933 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-94731191-4b92-47fb-a184-ab25161d23f8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:22.101157 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.wsgi [None req-94731191-4b92-47fb-a184-ab25161d23f8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] HTTP exception thrown: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-94731191-4b92-47fb-a184-ab25161d23f8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Request method failure captured: Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: request: GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d HTTP/1.1 Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept: application/json Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept-Encoding: identity Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Connection: close Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Length: 0 Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Type: application/json Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Host: 10.222.0.25 Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Openstack-System-Scope: None Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: User-Agent: python-urllib3/1.26.12 Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Auth-Token: gAAAAABjW6Sr9rY_ITV7HhEadU-8kVwknlsO0ILZxCejbC0tv-IXJnixigZ0yNp9FPJ0cREY_BiaBySs0qEXDy-B2HhLgm_7R6Uo426RjnExTKsP2_sj58xycvGdyeCTUklJQqy6gTOR8ZMSLaSiUs2I6gLtF7HzqTatq6lKFNab2p6V5DQOaCQ Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Id: None Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Name: None Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Identity-Status: Confirmed Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Is-Admin-Project: True Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Id: default Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Name: Default Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Id: f2fcce993e6b4388bde493f0702fd49d Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Name: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Role: member,reader Oct 28 09:50:22.104697 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Roles: member,reader Oct 28 09:50:22.106059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f2fcce993e6b4388bde493f0702fd49d"}]}] Oct 28 09:50:22.106059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:50:22.106059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Id: f2fcce993e6b4388bde493f0702fd49d Oct 28 09:50:22.106059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Name: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:50:22.106059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User: tempest-AttachVolumeTestJSON-996475968-project Oct 28 09:50:22.106059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Id: default Oct 28 09:50:22.106059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Name: Default Oct 28 09:50:22.106059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Id: a4237ca76acd4f7483aaf66e1f5d2eea Oct 28 09:50:22.106059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Name: tempest-AttachVolumeTestJSON-996475968-project Oct 28 09:50:22.106059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: method: > Oct 28 09:50:22.106059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: exception: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:50:22.106059 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: action_args: {'id': 'b981eca6-4d6f-4e48-bf62-016833f2885d'} Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:50:22.106989 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:50:22.108443 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:50:22.109658 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-94731191-4b92-47fb-a184-ab25161d23f8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Returning 404 to user: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. {{(pid=127029) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:50:22.109658 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-94731191-4b92-47fb-a184-ab25161d23f8 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 404 len: 111 microversion: 2.1 time: 0.117744 Oct 28 09:50:22.115810 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 384/750] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:21 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 111 bytes in 126 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:50:22.121580 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d2b21486-b88c-4bc4-9aff-f7f601bde2ba tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '.version_select at 0x7f271a395af0>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:22.150278 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d2b21486-b88c-4bc4-9aff-f7f601bde2ba tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1262841334" status: 202 len: 0 microversion: 2.1 time: 0.033329 Oct 28 09:50:22.156148 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 367/751] 10.222.0.25 () {58 vars in 1221 bytes} [Fri Oct 28 09:50:22 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1262841334 => generated 0 bytes in 35 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 09:50:24.141586 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-edf31c81-0fba-4513-bbc9-3c609955fabf req-084e287b-fe78-4ba4-a9b8-3486d23ecb4c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b68e375d-f7ab-4d35-a8c7-081596d0e6e2", "tag": "533de8d8-0438-4dd2-946d-f12312377670"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:24.154147 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-edf31c81-0fba-4513-bbc9-3c609955fabf req-084e287b-fe78-4ba4-a9b8-3486d23ecb4c service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:24.154838 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-edf31c81-0fba-4513-bbc9-3c609955fabf req-084e287b-fe78-4ba4-a9b8-3486d23ecb4c service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:24.155437 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-edf31c81-0fba-4513-bbc9-3c609955fabf req-084e287b-fe78-4ba4-a9b8-3486d23ecb4c service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:24.173800 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-edf31c81-0fba-4513-bbc9-3c609955fabf req-084e287b-fe78-4ba4-a9b8-3486d23ecb4c service nova] Creating event network-changed:533de8d8-0438-4dd2-946d-f12312377670 for instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 on hv-cinder-zed-3 Oct 28 09:50:24.182581 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-edf31c81-0fba-4513-bbc9-3c609955fabf req-084e287b-fe78-4ba4-a9b8-3486d23ecb4c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046721 Oct 28 09:50:24.184601 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 385/752] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:50:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:50:24.957345 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5da93bea-70a3-455e-8e7d-9afeb7649310 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Action: 'create', calling method: .version_select at 0x7f271a3229d0>, body: {"keypair": {"name": "tempest-keypair-1965089747", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBhMwvI1DzSB3lQaEFiXgTpYRdeNaOY9L3RYnxI37LFelBFSZuGSamFrkD0qBDEyN1if0yoCnYBomk0/mhV7hyrRQ8wRVXgHxJAnGmRLDeDpk4Z7emxZaaU93ILx47W2/Q=="}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:25.048343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-5da93bea-70a3-455e-8e7d-9afeb7649310 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Getting quotas for project f2fcce993e6b4388bde493f0702fd49d. Resources: {'key_pairs'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:50:25.082451 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-5da93bea-70a3-455e-8e7d-9afeb7649310 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Getting quotas for user a4237ca76acd4f7483aaf66e1f5d2eea and project f2fcce993e6b4388bde493f0702fd49d. Resources: {'key_pairs'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:50:25.160356 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5da93bea-70a3-455e-8e7d-9afeb7649310 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.211810 Oct 28 09:50:25.161783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 368/753] 10.222.0.25 () {60 vars in 1159 bytes} [Fri Oct 28 09:50:24 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:50:26.598845 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-00da92e3-38c2-4ea2-82b0-2de10d6b64c0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:26.603378 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-00da92e3-38c2-4ea2-82b0-2de10d6b64c0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:26.621784 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-00da92e3-38c2-4ea2-82b0-2de10d6b64c0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:26.624061 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-00da92e3-38c2-4ea2-82b0-2de10d6b64c0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:26.625252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-00da92e3-38c2-4ea2-82b0-2de10d6b64c0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:26.628968 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-00da92e3-38c2-4ea2-82b0-2de10d6b64c0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:26.630978 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-00da92e3-38c2-4ea2-82b0-2de10d6b64c0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:26.632079 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-00da92e3-38c2-4ea2-82b0-2de10d6b64c0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:26.778380 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.objects.instance [None req-00da92e3-38c2-4ea2-82b0-2de10d6b64c0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lazy-loading 'services' on Instance uuid 4891b63b-eb08-448d-a640-f9579c8140ff {{(pid=127029) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:50:26.874112 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-00da92e3-38c2-4ea2-82b0-2de10d6b64c0 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "DELETE /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-volume_attachments/855268e6-c6f2-4ab9-a6e6-79c92b1e0b32" status: 202 len: 0 microversion: 2.1 time: 0.284757 Oct 28 09:50:26.878855 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 386/754] 10.222.0.25 () {58 vars in 1416 bytes} [Fri Oct 28 09:50:26 2022] DELETE /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-volume_attachments/855268e6-c6f2-4ab9-a6e6-79c92b1e0b32 => generated 0 bytes in 291 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:50:26.894002 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f2048c34-124a-45de-977c-778094b381ba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:26.906381 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f2048c34-124a-45de-977c-778094b381ba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:26.926819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f2048c34-124a-45de-977c-778094b381ba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:26.927714 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f2048c34-124a-45de-977c-778094b381ba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:26.928569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f2048c34-124a-45de-977c-778094b381ba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:26.929561 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f2048c34-124a-45de-977c-778094b381ba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:26.930304 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f2048c34-124a-45de-977c-778094b381ba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:26.930983 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f2048c34-124a-45de-977c-778094b381ba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:26.983408 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f2048c34-124a-45de-977c-778094b381ba tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.100786 Oct 28 09:50:26.990818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 369/755] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:50:26 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-volume_attachments => generated 197 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:50:28.019115 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-d21ded98-6e9e-46f7-9836-cef0a58cb9b4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:28.035386 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-d21ded98-6e9e-46f7-9836-cef0a58cb9b4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:28.064421 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d21ded98-6e9e-46f7-9836-cef0a58cb9b4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:28.067267 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d21ded98-6e9e-46f7-9836-cef0a58cb9b4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:28.068654 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d21ded98-6e9e-46f7-9836-cef0a58cb9b4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:28.070021 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d21ded98-6e9e-46f7-9836-cef0a58cb9b4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:28.070915 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d21ded98-6e9e-46f7-9836-cef0a58cb9b4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:28.072779 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-d21ded98-6e9e-46f7-9836-cef0a58cb9b4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:28.144405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1438384038", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "1d4760fa-4445-40fa-b7a4-b3e4124ac823"}], "security_groups": [{"name": "tempest-securitygroup--1294868358"}], "key_name": "tempest-keypair-1965089747", "user_data": "}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:28.447588 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.network.neutron [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] validate_networks() for [('1d4760fa-4445-40fa-b7a4-b3e4124ac823', None, None, None, None, None)] {{(pid=127030) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:50:28.550793 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-d21ded98-6e9e-46f7-9836-cef0a58cb9b4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.556572 Oct 28 09:50:28.552344 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 387/756] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:50:27 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff/os-volume_attachments => generated 25 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 09:50:28.742814 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-b37058a3-d3b0-4d62-baa3-d71bdd51ca0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:28.743734 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b37058a3-d3b0-4d62-baa3-d71bdd51ca0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:28.756412 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b37058a3-d3b0-4d62-baa3-d71bdd51ca0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:28.757781 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b37058a3-d3b0-4d62-baa3-d71bdd51ca0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:28.761344 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b37058a3-d3b0-4d62-baa3-d71bdd51ca0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:28.764335 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b37058a3-d3b0-4d62-baa3-d71bdd51ca0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:28.767222 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b37058a3-d3b0-4d62-baa3-d71bdd51ca0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:28.768498 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-b37058a3-d3b0-4d62-baa3-d71bdd51ca0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:28.831850 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-b37058a3-d3b0-4d62-baa3-d71bdd51ca0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Going to try to terminate instance {{(pid=127029) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:50:28.867994 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:28.948544 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-b37058a3-d3b0-4d62-baa3-d71bdd51ca0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "DELETE /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 204 len: 0 microversion: 2.1 time: 0.213950 Oct 28 09:50:28.949933 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 388/757] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:50:28 2022] DELETE /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 0 bytes in 217 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 28 09:50:28.963400 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-cdaea5b8-2fe2-4e80-8bf5-68f86a37d25e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:28.966778 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-cdaea5b8-2fe2-4e80-8bf5-68f86a37d25e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:28.974968 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cdaea5b8-2fe2-4e80-8bf5-68f86a37d25e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:28.976143 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cdaea5b8-2fe2-4e80-8bf5-68f86a37d25e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:28.976835 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cdaea5b8-2fe2-4e80-8bf5-68f86a37d25e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:28.978594 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cdaea5b8-2fe2-4e80-8bf5-68f86a37d25e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:28.979327 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cdaea5b8-2fe2-4e80-8bf5-68f86a37d25e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:28.980183 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-cdaea5b8-2fe2-4e80-8bf5-68f86a37d25e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:29.379421 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-cdaea5b8-2fe2-4e80-8bf5-68f86a37d25e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:29.400109 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-cdaea5b8-2fe2-4e80-8bf5-68f86a37d25e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1690 microversion: 2.1 time: 0.436264 Oct 28 09:50:29.401470 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 389/758] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:28 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1690 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:50:29.590678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Flavor limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:50:29.591478 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Image limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:50:29.594222 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Flavor pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:50:29.595623 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Image pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:50:29.596340 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:50:29.614073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:29.614992 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:29.615799 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:29.617253 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:29.618124 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:29.618963 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:29.655455 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Getting quotas for project f2fcce993e6b4388bde493f0702fd49d. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:50:29.664358 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Getting quotas for user a4237ca76acd4f7483aaf66e1f5d2eea and project f2fcce993e6b4388bde493f0702fd49d. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:50:29.689057 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Going to run 1 instances... {{(pid=127030) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:50:29.694791 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127030) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:50:29.802607 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9333cda5-a035-4e20-8038-983f6aabce00 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.666965 Oct 28 09:50:29.810621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 370/759] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:50:28 2022] POST /compute/v2.1/servers => generated 402 bytes in 1677 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 09:50:29.829570 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-338dc232-99f2-4526-857e-2b31623beeb3 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:29.834073 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-338dc232-99f2-4526-857e-2b31623beeb3 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:30.085570 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-338dc232-99f2-4526-857e-2b31623beeb3 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:30.099367 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-338dc232-99f2-4526-857e-2b31623beeb3 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1238 microversion: 2.1 time: 0.281909 Oct 28 09:50:30.100669 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 390/760] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:29 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1238 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:30.443397 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6f111915-a4ed-493a-b8ee-fc05fbaf6e74 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:30.447713 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6f111915-a4ed-493a-b8ee-fc05fbaf6e74 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:30.472209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f111915-a4ed-493a-b8ee-fc05fbaf6e74 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:30.473230 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f111915-a4ed-493a-b8ee-fc05fbaf6e74 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:30.475910 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f111915-a4ed-493a-b8ee-fc05fbaf6e74 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:30.479587 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f111915-a4ed-493a-b8ee-fc05fbaf6e74 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:30.481241 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f111915-a4ed-493a-b8ee-fc05fbaf6e74 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:30.482573 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f111915-a4ed-493a-b8ee-fc05fbaf6e74 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:30.769584 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6f111915-a4ed-493a-b8ee-fc05fbaf6e74 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:30.781263 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6f111915-a4ed-493a-b8ee-fc05fbaf6e74 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1690 microversion: 2.1 time: 0.346040 Oct 28 09:50:30.784639 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 371/761] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:30 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1690 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:50:31.142081 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-459556d2-372e-489c-90f9-3bfa9b207372 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:31.144502 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-459556d2-372e-489c-90f9-3bfa9b207372 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:31.152498 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-459556d2-372e-489c-90f9-3bfa9b207372 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:31.153883 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-459556d2-372e-489c-90f9-3bfa9b207372 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:31.154487 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-459556d2-372e-489c-90f9-3bfa9b207372 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:31.155562 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-459556d2-372e-489c-90f9-3bfa9b207372 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:31.156111 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-459556d2-372e-489c-90f9-3bfa9b207372 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:31.156619 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-459556d2-372e-489c-90f9-3bfa9b207372 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:31.248808 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-459556d2-372e-489c-90f9-3bfa9b207372 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:31.264083 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-459556d2-372e-489c-90f9-3bfa9b207372 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1242 microversion: 2.1 time: 0.127895 Oct 28 09:50:31.265054 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 391/762] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:31 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1242 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:31.846688 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-fe087547-0fb5-4030-bdd7-3f807a4a2c08 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:31.848802 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-fe087547-0fb5-4030-bdd7-3f807a4a2c08 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:31.862497 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fe087547-0fb5-4030-bdd7-3f807a4a2c08 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:31.863348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fe087547-0fb5-4030-bdd7-3f807a4a2c08 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:31.864107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fe087547-0fb5-4030-bdd7-3f807a4a2c08 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:31.865574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fe087547-0fb5-4030-bdd7-3f807a4a2c08 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:31.866227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fe087547-0fb5-4030-bdd7-3f807a4a2c08 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:31.867042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fe087547-0fb5-4030-bdd7-3f807a4a2c08 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:32.074635 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-fe087547-0fb5-4030-bdd7-3f807a4a2c08 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:32.084481 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-fe087547-0fb5-4030-bdd7-3f807a4a2c08 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1690 microversion: 2.1 time: 0.251403 Oct 28 09:50:32.086951 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 372/763] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:31 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1690 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:32.289524 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-f4fe1fcb-8f1a-4d7e-ab0c-dc4fc91d024f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:32.290374 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-f4fe1fcb-8f1a-4d7e-ab0c-dc4fc91d024f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:32.296849 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=84,nova_api:DELETE=2,nova_api:INSERT=4,nova_api:UPDATE=2 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:32.301909 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f4fe1fcb-8f1a-4d7e-ab0c-dc4fc91d024f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:32.302948 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f4fe1fcb-8f1a-4d7e-ab0c-dc4fc91d024f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:32.303404 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f4fe1fcb-8f1a-4d7e-ab0c-dc4fc91d024f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:32.304341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f4fe1fcb-8f1a-4d7e-ab0c-dc4fc91d024f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:32.304850 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f4fe1fcb-8f1a-4d7e-ab0c-dc4fc91d024f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:32.305360 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-f4fe1fcb-8f1a-4d7e-ab0c-dc4fc91d024f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:32.381285 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-f4fe1fcb-8f1a-4d7e-ab0c-dc4fc91d024f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:32.390476 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-f4fe1fcb-8f1a-4d7e-ab0c-dc4fc91d024f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1242 microversion: 2.1 time: 0.105192 Oct 28 09:50:32.391609 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 392/764] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:32 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1242 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:50:33.114801 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3115b627-54ac-425b-8584-ffdddf3f167f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:33.115749 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3115b627-54ac-425b-8584-ffdddf3f167f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:33.133156 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3115b627-54ac-425b-8584-ffdddf3f167f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:33.133992 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3115b627-54ac-425b-8584-ffdddf3f167f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:33.134825 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3115b627-54ac-425b-8584-ffdddf3f167f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:33.136695 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3115b627-54ac-425b-8584-ffdddf3f167f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:33.142375 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3115b627-54ac-425b-8584-ffdddf3f167f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:33.143225 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3115b627-54ac-425b-8584-ffdddf3f167f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:33.155544 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=83 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:33.416395 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6a8c9422-d337-4486-81c5-b9a15ee094da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:33.418188 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6a8c9422-d337-4486-81c5-b9a15ee094da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:33.431493 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6a8c9422-d337-4486-81c5-b9a15ee094da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:33.432182 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6a8c9422-d337-4486-81c5-b9a15ee094da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:33.432713 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6a8c9422-d337-4486-81c5-b9a15ee094da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:33.433631 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6a8c9422-d337-4486-81c5-b9a15ee094da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:33.434234 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6a8c9422-d337-4486-81c5-b9a15ee094da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:33.434771 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6a8c9422-d337-4486-81c5-b9a15ee094da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:33.959446 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6a8c9422-d337-4486-81c5-b9a15ee094da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:34.005439 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6a8c9422-d337-4486-81c5-b9a15ee094da tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1290 microversion: 2.1 time: 0.592966 Oct 28 09:50:34.011553 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 393/765] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:33 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1290 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:34.456696 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3115b627-54ac-425b-8584-ffdddf3f167f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:34.474585 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3115b627-54ac-425b-8584-ffdddf3f167f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1690 microversion: 2.1 time: 1.364914 Oct 28 09:50:34.475656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 373/766] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:33 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1690 bytes in 1367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:35.035973 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-a04a6b0e-4660-43a7-a54c-0c30298b434b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:35.036806 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-a04a6b0e-4660-43a7-a54c-0c30298b434b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:35.063286 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a04a6b0e-4660-43a7-a54c-0c30298b434b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:35.067862 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a04a6b0e-4660-43a7-a54c-0c30298b434b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:35.070910 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a04a6b0e-4660-43a7-a54c-0c30298b434b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:35.075091 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a04a6b0e-4660-43a7-a54c-0c30298b434b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:35.076316 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a04a6b0e-4660-43a7-a54c-0c30298b434b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:35.080213 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-a04a6b0e-4660-43a7-a54c-0c30298b434b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:35.278936 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-a04a6b0e-4660-43a7-a54c-0c30298b434b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:35.295913 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-a04a6b0e-4660-43a7-a54c-0c30298b434b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1296 microversion: 2.1 time: 0.266199 Oct 28 09:50:35.298030 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 394/767] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:35 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1296 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:50:35.511009 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2e7413e8-e34c-44e8-aa32-afeaa3530bca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:35.512483 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2e7413e8-e34c-44e8-aa32-afeaa3530bca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:35.526131 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2e7413e8-e34c-44e8-aa32-afeaa3530bca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:35.527436 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2e7413e8-e34c-44e8-aa32-afeaa3530bca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:35.531030 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2e7413e8-e34c-44e8-aa32-afeaa3530bca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:35.532294 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2e7413e8-e34c-44e8-aa32-afeaa3530bca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:35.534934 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2e7413e8-e34c-44e8-aa32-afeaa3530bca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:35.535973 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2e7413e8-e34c-44e8-aa32-afeaa3530bca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:35.889875 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2e7413e8-e34c-44e8-aa32-afeaa3530bca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:35.901167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2e7413e8-e34c-44e8-aa32-afeaa3530bca tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1622 microversion: 2.1 time: 0.395752 Oct 28 09:50:35.902936 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 374/768] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:35 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1622 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:36.325707 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-85f6c384-07a3-429b-90d0-01602c57de8a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:36.326580 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-85f6c384-07a3-429b-90d0-01602c57de8a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:36.340540 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-85f6c384-07a3-429b-90d0-01602c57de8a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:36.343908 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-85f6c384-07a3-429b-90d0-01602c57de8a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:36.345015 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-85f6c384-07a3-429b-90d0-01602c57de8a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:36.356166 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-85f6c384-07a3-429b-90d0-01602c57de8a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:36.357252 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-85f6c384-07a3-429b-90d0-01602c57de8a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:36.357940 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-85f6c384-07a3-429b-90d0-01602c57de8a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:36.551000 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-85f6c384-07a3-429b-90d0-01602c57de8a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:36.572861 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-85f6c384-07a3-429b-90d0-01602c57de8a tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1296 microversion: 2.1 time: 0.255800 Oct 28 09:50:36.582853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 375/769] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:36 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1296 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:36.725473 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-da7bc1a5-44d2-48ec-a9b0-63dce77c342f req-719a3a6e-8af5-43a0-a871-49198ab00f30 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4891b63b-eb08-448d-a640-f9579c8140ff", "name": "network-vif-deleted", "tag": "6e78f30d-6812-43c6-910d-70f563ee8ad9"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:36.742534 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-da7bc1a5-44d2-48ec-a9b0-63dce77c342f req-719a3a6e-8af5-43a0-a871-49198ab00f30 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:36.746801 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-da7bc1a5-44d2-48ec-a9b0-63dce77c342f req-719a3a6e-8af5-43a0-a871-49198ab00f30 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:36.754684 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-da7bc1a5-44d2-48ec-a9b0-63dce77c342f req-719a3a6e-8af5-43a0-a871-49198ab00f30 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:36.819045 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-da7bc1a5-44d2-48ec-a9b0-63dce77c342f req-719a3a6e-8af5-43a0-a871-49198ab00f30 service nova] Creating event network-vif-deleted:6e78f30d-6812-43c6-910d-70f563ee8ad9 for instance 4891b63b-eb08-448d-a640-f9579c8140ff on hv-cinder-zed-3 Oct 28 09:50:36.827947 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-da7bc1a5-44d2-48ec-a9b0-63dce77c342f req-719a3a6e-8af5-43a0-a871-49198ab00f30 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.722579 Oct 28 09:50:36.829881 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 395/770] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:50:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 727 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:50:36.946326 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:36.947941 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:36.982462 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:36.985645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:36.988011 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:36.989830 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:36.994289 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:37.000114 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:37.182677 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:37.183555 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:37.184666 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:37.185755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lazy-loading 'fault' on Instance uuid 4891b63b-eb08-448d-a640-f9579c8140ff {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:50:37.327407 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:37.363005 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d0aa6283-7acb-48c8-ae8e-49f6588065cb tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 200 len: 1335 microversion: 2.1 time: 0.424399 Oct 28 09:50:37.367671 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 376/771] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:36 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 1335 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:37.614317 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-06b22a75-4606-4ea3-a4b3-9bb127275198 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:37.616553 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-06b22a75-4606-4ea3-a4b3-9bb127275198 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:37.639861 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-06b22a75-4606-4ea3-a4b3-9bb127275198 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:37.644230 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-06b22a75-4606-4ea3-a4b3-9bb127275198 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:37.645275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-06b22a75-4606-4ea3-a4b3-9bb127275198 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:37.651454 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-06b22a75-4606-4ea3-a4b3-9bb127275198 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:37.652559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-06b22a75-4606-4ea3-a4b3-9bb127275198 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:37.657311 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-06b22a75-4606-4ea3-a4b3-9bb127275198 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:37.807731 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-f4009dcf-e0fd-4f3b-9d57-8a7ebf77f7b5 tempest-TestVolumeBootPattern-1984064024 tempest-TestVolumeBootPattern-1984064024-project] Action: 'create', calling method: .version_select at 0x7f271a2f35e0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-196867844", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDJQ91e8vWuAdcJzoy/Q6quE1q52Bo9InxxzuwfIJU+MmJwwqe1ye0EhrWYfgRsyXJFOVht5/zIGLE1l+i8Tkm2Zpo0IbLtJf7JtljBvMlj4MFTeG1jcMmWGsrrGNoI+rQ=="}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:37.850112 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-f4009dcf-e0fd-4f3b-9d57-8a7ebf77f7b5 tempest-TestVolumeBootPattern-1984064024 tempest-TestVolumeBootPattern-1984064024-project] Getting quotas for project 6d071a4f24b541d4b5d49017707e0788. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:50:37.860837 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.quota [None req-f4009dcf-e0fd-4f3b-9d57-8a7ebf77f7b5 tempest-TestVolumeBootPattern-1984064024 tempest-TestVolumeBootPattern-1984064024-project] Getting quotas for user a0e517bf245e44b99fd23325307ba2cb and project 6d071a4f24b541d4b5d49017707e0788. Resources: {'key_pairs'} {{(pid=127029) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:50:37.908823 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-f4009dcf-e0fd-4f3b-9d57-8a7ebf77f7b5 tempest-TestVolumeBootPattern-1984064024 tempest-TestVolumeBootPattern-1984064024-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.454949 Oct 28 09:50:37.910278 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 396/772] 10.222.0.25 () {60 vars in 1159 bytes} [Fri Oct 28 09:50:37 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:50:38.039404 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-06b22a75-4606-4ea3-a4b3-9bb127275198 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:38.075366 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-06b22a75-4606-4ea3-a4b3-9bb127275198 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1364 microversion: 2.1 time: 0.469350 Oct 28 09:50:38.078123 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 377/773] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:37 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1364 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:38.404351 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-fe12e3f1-ba10-4bc6-b288-6844aaa88284 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:38.411331 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-fe12e3f1-ba10-4bc6-b288-6844aaa88284 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 4891b63b-eb08-448d-a640-f9579c8140ff] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:38.449651 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fe12e3f1-ba10-4bc6-b288-6844aaa88284 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:38.452614 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fe12e3f1-ba10-4bc6-b288-6844aaa88284 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:38.454149 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fe12e3f1-ba10-4bc6-b288-6844aaa88284 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:38.456524 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fe12e3f1-ba10-4bc6-b288-6844aaa88284 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:38.457377 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fe12e3f1-ba10-4bc6-b288-6844aaa88284 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:38.458035 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-fe12e3f1-ba10-4bc6-b288-6844aaa88284 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:38.493867 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.wsgi [None req-fe12e3f1-ba10-4bc6-b288-6844aaa88284 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] HTTP exception thrown: Instance 4891b63b-eb08-448d-a640-f9579c8140ff could not be found. Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-fe12e3f1-ba10-4bc6-b288-6844aaa88284 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Request method failure captured: Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: request: GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff HTTP/1.1 Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept: application/json Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Accept-Encoding: identity Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Connection: close Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Length: 0 Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Content-Type: application/json Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Host: 10.222.0.25 Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Openstack-System-Scope: None Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: User-Agent: python-urllib3/1.26.12 Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Auth-Token: gAAAAABjW6Sjq718s-6y9fBFjvzR8wDyF_4AlWYEmdP4icnuJ0s_5turdxYbjJNH08Q4u87xp47MpEwc_KXLuW4AwBVGzdBvRRVA1GVRVASenhMGF1IGhESh28OTGp6LsG6ZaKxtUJblyS6zcH0d96QlXz7dkrheK75c4DcQ-AxWdIWnFfDpBXE Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Id: None Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Domain-Name: None Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Identity-Status: Confirmed Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Is-Admin-Project: True Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Id: default Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Domain-Name: Default Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Id: 227ac6c4e642467ca3b9e0bf7181d5c9 Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Role: member,reader Oct 28 09:50:38.500821 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Roles: member,reader Oct 28 09:50:38.504891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/227ac6c4e642467ca3b9e0bf7181d5c9"}]}] Oct 28 09:50:38.504891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:50:38.504891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Id: 227ac6c4e642467ca3b9e0bf7181d5c9 Oct 28 09:50:38.504891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:50:38.504891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User: tempest-ServerRescueNegativeTestJSON-1296287768-project Oct 28 09:50:38.504891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Id: default Oct 28 09:50:38.504891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Domain-Name: Default Oct 28 09:50:38.504891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Id: ed9a4c24cbfb4689b7f984219c7503a4 Oct 28 09:50:38.504891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1296287768-project Oct 28 09:50:38.504891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: method: > Oct 28 09:50:38.504891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: exception: Instance 4891b63b-eb08-448d-a640-f9579c8140ff could not be found. Oct 28 09:50:38.504891 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: action_args: {'id': '4891b63b-eb08-448d-a640-f9579c8140ff'} Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4891b63b-eb08-448d-a640-f9579c8140ff could not be found. Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:50:38.506605 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4891b63b-eb08-448d-a640-f9579c8140ff could not be found. Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4891b63b-eb08-448d-a640-f9579c8140ff could not be found. Oct 28 09:50:38.509252 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: ERROR nova.api.openstack.wsgi Oct 28 09:50:38.513137 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-fe12e3f1-ba10-4bc6-b288-6844aaa88284 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Returning 404 to user: Instance 4891b63b-eb08-448d-a640-f9579c8140ff could not be found. {{(pid=127029) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:50:38.516611 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-fe12e3f1-ba10-4bc6-b288-6844aaa88284 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff" status: 404 len: 111 microversion: 2.1 time: 0.122958 Oct 28 09:50:38.518554 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 397/774] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:38 2022] GET /compute/v2.1/servers/4891b63b-eb08-448d-a640-f9579c8140ff => generated 111 bytes in 128 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 09:50:38.536181 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ad629a9b-ec5d-4214-a88d-b7dbf035f697 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '.version_select at 0x7f271a3595e0>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:38.599776 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ad629a9b-ec5d-4214-a88d-b7dbf035f697 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-757421500" status: 202 len: 0 microversion: 2.1 time: 0.074344 Oct 28 09:50:38.608224 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 378/775] 10.222.0.25 () {58 vars in 1218 bytes} [Fri Oct 28 09:50:38 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-757421500 => generated 0 bytes in 84 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:50:39.113920 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6828e50a-da88-4b7d-92e2-12d49ed73dd1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:39.115033 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6828e50a-da88-4b7d-92e2-12d49ed73dd1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:39.124451 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6828e50a-da88-4b7d-92e2-12d49ed73dd1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:39.124941 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6828e50a-da88-4b7d-92e2-12d49ed73dd1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:39.125831 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6828e50a-da88-4b7d-92e2-12d49ed73dd1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:39.126276 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6828e50a-da88-4b7d-92e2-12d49ed73dd1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:39.126758 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6828e50a-da88-4b7d-92e2-12d49ed73dd1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:39.127186 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6828e50a-da88-4b7d-92e2-12d49ed73dd1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:39.135124 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-8c7398b5-112c-47cf-81ab-e6a6b8a894c1 req-dcafaf0f-1b93-40b0-812e-e98c72c8b4d5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c28f1b03-788f-42f6-8577-9c81bffc41ae", "tag": "1fff934b-bb90-4e1c-b8bd-845cc947058d"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:39.159922 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-8c7398b5-112c-47cf-81ab-e6a6b8a894c1 req-dcafaf0f-1b93-40b0-812e-e98c72c8b4d5 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:39.167344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-8c7398b5-112c-47cf-81ab-e6a6b8a894c1 req-dcafaf0f-1b93-40b0-812e-e98c72c8b4d5 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:39.171140 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-8c7398b5-112c-47cf-81ab-e6a6b8a894c1 req-dcafaf0f-1b93-40b0-812e-e98c72c8b4d5 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:39.221735 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-8c7398b5-112c-47cf-81ab-e6a6b8a894c1 req-dcafaf0f-1b93-40b0-812e-e98c72c8b4d5 service nova] Creating event network-changed:1fff934b-bb90-4e1c-b8bd-845cc947058d for instance c28f1b03-788f-42f6-8577-9c81bffc41ae on hv-cinder-zed-3 Oct 28 09:50:39.246433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-8c7398b5-112c-47cf-81ab-e6a6b8a894c1 req-dcafaf0f-1b93-40b0-812e-e98c72c8b4d5 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.115176 Oct 28 09:50:39.254011 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 379/776] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:50:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:50:39.672753 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6828e50a-da88-4b7d-92e2-12d49ed73dd1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:39.696705 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6828e50a-da88-4b7d-92e2-12d49ed73dd1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1364 microversion: 2.1 time: 0.586797 Oct 28 09:50:39.716118 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 398/777] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:39 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1364 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:40.729206 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4b062948-a038-4ef6-a9bc-5e15562a0ae7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:40.731340 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4b062948-a038-4ef6-a9bc-5e15562a0ae7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:40.735302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:40.736386 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=10 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:40.749000 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b062948-a038-4ef6-a9bc-5e15562a0ae7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:40.750145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b062948-a038-4ef6-a9bc-5e15562a0ae7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:40.751304 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b062948-a038-4ef6-a9bc-5e15562a0ae7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:40.752903 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b062948-a038-4ef6-a9bc-5e15562a0ae7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:40.753933 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b062948-a038-4ef6-a9bc-5e15562a0ae7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:40.756089 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b062948-a038-4ef6-a9bc-5e15562a0ae7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:41.075796 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-4b062948-a038-4ef6-a9bc-5e15562a0ae7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:41.098279 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4b062948-a038-4ef6-a9bc-5e15562a0ae7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1364 microversion: 2.1 time: 0.377469 Oct 28 09:50:41.110573 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 380/778] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:40 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1364 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:42.128484 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-8a330aa1-c471-45ad-a7d8-1fe474491a85 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:42.129334 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-8a330aa1-c471-45ad-a7d8-1fe474491a85 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:42.141450 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8a330aa1-c471-45ad-a7d8-1fe474491a85 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:42.142276 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8a330aa1-c471-45ad-a7d8-1fe474491a85 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:42.142770 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8a330aa1-c471-45ad-a7d8-1fe474491a85 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:42.143544 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8a330aa1-c471-45ad-a7d8-1fe474491a85 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:42.143993 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8a330aa1-c471-45ad-a7d8-1fe474491a85 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:42.144426 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-8a330aa1-c471-45ad-a7d8-1fe474491a85 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:42.334966 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-8a330aa1-c471-45ad-a7d8-1fe474491a85 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:42.347017 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-8a330aa1-c471-45ad-a7d8-1fe474491a85 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1364 microversion: 2.1 time: 0.222828 Oct 28 09:50:42.347915 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 399/779] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:42 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1364 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:50:42.525475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3b155268-2dc6-4470-85ad-c25e011a610e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:42.526100 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3b155268-2dc6-4470-85ad-c25e011a610e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:42.536608 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b155268-2dc6-4470-85ad-c25e011a610e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:42.537108 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b155268-2dc6-4470-85ad-c25e011a610e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:42.537558 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b155268-2dc6-4470-85ad-c25e011a610e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:42.538318 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b155268-2dc6-4470-85ad-c25e011a610e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:42.539020 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b155268-2dc6-4470-85ad-c25e011a610e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:42.539451 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b155268-2dc6-4470-85ad-c25e011a610e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:42.569872 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3b155268-2dc6-4470-85ad-c25e011a610e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Going to try to terminate instance {{(pid=127030) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:50:42.664793 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3b155268-2dc6-4470-85ad-c25e011a610e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "DELETE /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 204 len: 0 microversion: 2.1 time: 0.142915 Oct 28 09:50:42.665615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 381/780] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:50:42 2022] DELETE /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:50:42.676054 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-1c324f6c-10d4-4cca-bad2-50f0455b568a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:42.676793 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-1c324f6c-10d4-4cca-bad2-50f0455b568a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:42.683393 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1c324f6c-10d4-4cca-bad2-50f0455b568a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:42.683966 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1c324f6c-10d4-4cca-bad2-50f0455b568a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:42.684403 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1c324f6c-10d4-4cca-bad2-50f0455b568a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:42.685119 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1c324f6c-10d4-4cca-bad2-50f0455b568a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:42.685576 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1c324f6c-10d4-4cca-bad2-50f0455b568a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:42.685995 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1c324f6c-10d4-4cca-bad2-50f0455b568a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:42.857470 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-1c324f6c-10d4-4cca-bad2-50f0455b568a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:42.867601 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-1c324f6c-10d4-4cca-bad2-50f0455b568a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1508 microversion: 2.1 time: 0.196175 Oct 28 09:50:42.868342 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 400/781] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:42 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1508 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:43.372620 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-64e76b42-05de-4430-8ffe-95567dc03104 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:43.373450 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-64e76b42-05de-4430-8ffe-95567dc03104 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:43.380311 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64e76b42-05de-4430-8ffe-95567dc03104 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:43.380813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64e76b42-05de-4430-8ffe-95567dc03104 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:43.381222 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64e76b42-05de-4430-8ffe-95567dc03104 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:43.381925 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64e76b42-05de-4430-8ffe-95567dc03104 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:43.382370 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64e76b42-05de-4430-8ffe-95567dc03104 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:43.382818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64e76b42-05de-4430-8ffe-95567dc03104 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:43.576206 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-64e76b42-05de-4430-8ffe-95567dc03104 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:43.586672 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-64e76b42-05de-4430-8ffe-95567dc03104 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1364 microversion: 2.1 time: 0.218484 Oct 28 09:50:43.587270 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 382/782] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:43 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1364 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:43.895149 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-59abf2a2-9bee-42d5-9fee-42095addeac6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:43.895966 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-59abf2a2-9bee-42d5-9fee-42095addeac6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:43.902651 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-59abf2a2-9bee-42d5-9fee-42095addeac6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:43.903151 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-59abf2a2-9bee-42d5-9fee-42095addeac6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:43.903592 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-59abf2a2-9bee-42d5-9fee-42095addeac6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:43.904318 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-59abf2a2-9bee-42d5-9fee-42095addeac6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:43.904776 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-59abf2a2-9bee-42d5-9fee-42095addeac6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:43.905229 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-59abf2a2-9bee-42d5-9fee-42095addeac6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:44.074832 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-59abf2a2-9bee-42d5-9fee-42095addeac6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:44.083933 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-59abf2a2-9bee-42d5-9fee-42095addeac6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1508 microversion: 2.1 time: 0.193072 Oct 28 09:50:44.084526 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 401/783] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:43 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1508 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:44.615652 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b6f6e2b0-6c6c-4813-b23c-7ad10255beba tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:44.616708 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b6f6e2b0-6c6c-4813-b23c-7ad10255beba tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:44.624101 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6f6e2b0-6c6c-4813-b23c-7ad10255beba tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:44.624601 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6f6e2b0-6c6c-4813-b23c-7ad10255beba tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:44.625048 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6f6e2b0-6c6c-4813-b23c-7ad10255beba tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:44.626353 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6f6e2b0-6c6c-4813-b23c-7ad10255beba tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:44.626844 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6f6e2b0-6c6c-4813-b23c-7ad10255beba tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:44.627284 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6f6e2b0-6c6c-4813-b23c-7ad10255beba tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:44.808295 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b6f6e2b0-6c6c-4813-b23c-7ad10255beba tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:44.824829 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b6f6e2b0-6c6c-4813-b23c-7ad10255beba tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1364 microversion: 2.1 time: 0.209160 Oct 28 09:50:44.825874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 383/784] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:44 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1364 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:50:45.114080 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-1b094de5-93e8-4513-85fa-7a2d2961165a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:45.115632 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-1b094de5-93e8-4513-85fa-7a2d2961165a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:45.126297 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1b094de5-93e8-4513-85fa-7a2d2961165a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:45.127347 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1b094de5-93e8-4513-85fa-7a2d2961165a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:45.128020 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1b094de5-93e8-4513-85fa-7a2d2961165a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:45.129613 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1b094de5-93e8-4513-85fa-7a2d2961165a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:45.131174 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1b094de5-93e8-4513-85fa-7a2d2961165a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:45.132289 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-1b094de5-93e8-4513-85fa-7a2d2961165a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:45.688756 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-1b094de5-93e8-4513-85fa-7a2d2961165a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:45.703234 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-1b094de5-93e8-4513-85fa-7a2d2961165a tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1508 microversion: 2.1 time: 0.594906 Oct 28 09:50:45.718720 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 402/785] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:45 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1508 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:45.858649 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7ab00976-f52d-4580-bac5-54c23d8ec79c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:45.860508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7ab00976-f52d-4580-bac5-54c23d8ec79c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:45.867935 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7ab00976-f52d-4580-bac5-54c23d8ec79c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:45.868424 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7ab00976-f52d-4580-bac5-54c23d8ec79c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:45.868884 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7ab00976-f52d-4580-bac5-54c23d8ec79c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:45.869810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7ab00976-f52d-4580-bac5-54c23d8ec79c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:45.870358 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7ab00976-f52d-4580-bac5-54c23d8ec79c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:45.871012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7ab00976-f52d-4580-bac5-54c23d8ec79c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:45.906616 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-cc5dffe0-13ac-4ea9-9d2d-d8b9a3d27166 req-54b4b83a-6e97-488e-b017-2e7a24fcbc97 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7", "name": "network-vif-deleted", "tag": "a69a90a6-23e5-4976-90d8-d36b10ff6b35"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:45.932058 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-cc5dffe0-13ac-4ea9-9d2d-d8b9a3d27166 req-54b4b83a-6e97-488e-b017-2e7a24fcbc97 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:45.932999 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-cc5dffe0-13ac-4ea9-9d2d-d8b9a3d27166 req-54b4b83a-6e97-488e-b017-2e7a24fcbc97 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:45.934267 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-cc5dffe0-13ac-4ea9-9d2d-d8b9a3d27166 req-54b4b83a-6e97-488e-b017-2e7a24fcbc97 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:45.963139 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-cc5dffe0-13ac-4ea9-9d2d-d8b9a3d27166 req-54b4b83a-6e97-488e-b017-2e7a24fcbc97 service nova] Creating event network-vif-deleted:a69a90a6-23e5-4976-90d8-d36b10ff6b35 for instance cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 on hv-cinder-zed-3 Oct 28 09:50:45.972420 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-cc5dffe0-13ac-4ea9-9d2d-d8b9a3d27166 req-54b4b83a-6e97-488e-b017-2e7a24fcbc97 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074008 Oct 28 09:50:45.973371 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 403/786] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:50:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:50:46.108701 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7ab00976-f52d-4580-bac5-54c23d8ec79c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:46.121879 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7ab00976-f52d-4580-bac5-54c23d8ec79c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1364 microversion: 2.1 time: 0.268006 Oct 28 09:50:46.122580 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 384/787] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:45 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1364 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:46.531259 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-e695a342-c208-4d76-b3e8-2c0d36e02eab req-ba6c2171-ffff-4513-963c-6573647e7c99 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c28f1b03-788f-42f6-8577-9c81bffc41ae", "name": "network-vif-plugged", "status": "completed", "tag": "1fff934b-bb90-4e1c-b8bd-845cc947058d"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:46.543707 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-e695a342-c208-4d76-b3e8-2c0d36e02eab req-ba6c2171-ffff-4513-963c-6573647e7c99 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:46.544178 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-e695a342-c208-4d76-b3e8-2c0d36e02eab req-ba6c2171-ffff-4513-963c-6573647e7c99 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:46.544785 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-e695a342-c208-4d76-b3e8-2c0d36e02eab req-ba6c2171-ffff-4513-963c-6573647e7c99 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:46.558385 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-e695a342-c208-4d76-b3e8-2c0d36e02eab req-ba6c2171-ffff-4513-963c-6573647e7c99 service nova] Creating event network-vif-plugged:1fff934b-bb90-4e1c-b8bd-845cc947058d for instance c28f1b03-788f-42f6-8577-9c81bffc41ae on hv-cinder-zed-3 Oct 28 09:50:46.563720 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-e695a342-c208-4d76-b3e8-2c0d36e02eab req-ba6c2171-ffff-4513-963c-6573647e7c99 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100683 Oct 28 09:50:46.564314 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 404/788] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:50:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:50:46.732037 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:46.733094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:46.739982 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:46.740482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:46.740911 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:46.741619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:46.742285 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:46.742821 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:46.792889 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:46.793413 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:46.793989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:46.794540 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lazy-loading 'fault' on Instance uuid cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:50:46.851638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:46.865125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5166170b-2e0f-4550-b7d7-a7a6f7b38a45 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 200 len: 1312 microversion: 2.1 time: 0.139181 Oct 28 09:50:46.866206 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 385/789] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:46 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 1312 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:47.163173 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-2a2e9eee-64eb-49f8-ac64-147f36bc7d49 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:47.163912 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-2a2e9eee-64eb-49f8-ac64-147f36bc7d49 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:47.171037 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2a2e9eee-64eb-49f8-ac64-147f36bc7d49 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:47.171532 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2a2e9eee-64eb-49f8-ac64-147f36bc7d49 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:47.171966 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2a2e9eee-64eb-49f8-ac64-147f36bc7d49 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:47.172684 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2a2e9eee-64eb-49f8-ac64-147f36bc7d49 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:47.173138 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2a2e9eee-64eb-49f8-ac64-147f36bc7d49 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:47.173562 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-2a2e9eee-64eb-49f8-ac64-147f36bc7d49 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:47.338530 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-2a2e9eee-64eb-49f8-ac64-147f36bc7d49 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:47.346690 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-2a2e9eee-64eb-49f8-ac64-147f36bc7d49 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1364 microversion: 2.1 time: 0.191765 Oct 28 09:50:47.347118 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 405/790] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:47 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1364 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:47.895682 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-256122f8-6f2e-4e30-b169-aa765238d45e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:47.896420 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-256122f8-6f2e-4e30-b169-aa765238d45e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:47.899219 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=83,nova_api:DELETE=2,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:47.906839 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-256122f8-6f2e-4e30-b169-aa765238d45e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:47.907409 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-256122f8-6f2e-4e30-b169-aa765238d45e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:47.907889 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-256122f8-6f2e-4e30-b169-aa765238d45e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:47.908656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-256122f8-6f2e-4e30-b169-aa765238d45e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:47.909186 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-256122f8-6f2e-4e30-b169-aa765238d45e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:47.909647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-256122f8-6f2e-4e30-b169-aa765238d45e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:47.928813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-256122f8-6f2e-4e30-b169-aa765238d45e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] HTTP exception thrown: Instance cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 could not be found. Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-256122f8-6f2e-4e30-b169-aa765238d45e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Request method failure captured: Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 HTTP/1.1 Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sjq718s-6y9fBFjvzR8wDyF_4AlWYEmdP4icnuJ0s_5turdxYbjJNH08Q4u87xp47MpEwc_KXLuW4AwBVGzdBvRRVA1GVRVASenhMGF1IGhESh28OTGp6LsG6ZaKxtUJblyS6zcH0d96QlXz7dkrheK75c4DcQ-AxWdIWnFfDpBXE Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 227ac6c4e642467ca3b9e0bf7181d5c9 Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:50:47.930442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:50:47.932807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/227ac6c4e642467ca3b9e0bf7181d5c9"}]}] Oct 28 09:50:47.932807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:50:47.932807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 227ac6c4e642467ca3b9e0bf7181d5c9 Oct 28 09:50:47.932807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:50:47.932807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-ServerRescueNegativeTestJSON-1296287768-project Oct 28 09:50:47.932807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:50:47.932807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:50:47.932807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: ed9a4c24cbfb4689b7f984219c7503a4 Oct 28 09:50:47.932807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1296287768-project Oct 28 09:50:47.932807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:50:47.932807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 could not be found. Oct 28 09:50:47.932807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7'} Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 could not be found. Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:50:47.934086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 could not be found. Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 could not be found. Oct 28 09:50:47.936084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:47.937946 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-256122f8-6f2e-4e30-b169-aa765238d45e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Returning 404 to user: Instance cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:50:47.937946 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-256122f8-6f2e-4e30-b169-aa765238d45e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7" status: 404 len: 111 microversion: 2.1 time: 0.041758 Oct 28 09:50:47.937946 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 386/791] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:47 2022] GET /compute/v2.1/servers/cd2cba4a-f8fd-4fe2-ae69-28430c1f7bc7 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:50:47.942481 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-5e6a8a8e-4c5c-4dee-bebe-7ba8de6874c4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:47.943100 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5e6a8a8e-4c5c-4dee-bebe-7ba8de6874c4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:47.945566 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell0:SELECT=6 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:47.951374 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e6a8a8e-4c5c-4dee-bebe-7ba8de6874c4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:47.951864 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e6a8a8e-4c5c-4dee-bebe-7ba8de6874c4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:47.952302 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e6a8a8e-4c5c-4dee-bebe-7ba8de6874c4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:47.953013 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e6a8a8e-4c5c-4dee-bebe-7ba8de6874c4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:47.953508 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e6a8a8e-4c5c-4dee-bebe-7ba8de6874c4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:47.954041 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-5e6a8a8e-4c5c-4dee-bebe-7ba8de6874c4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:47.983737 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-5e6a8a8e-4c5c-4dee-bebe-7ba8de6874c4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Going to try to terminate instance {{(pid=127029) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:50:48.099152 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-5e6a8a8e-4c5c-4dee-bebe-7ba8de6874c4 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "DELETE /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 204 len: 0 microversion: 2.1 time: 0.160166 Oct 28 09:50:48.099951 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 406/792] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:50:47 2022] DELETE /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:50:48.108876 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a174f840-ef84-48a7-999e-228c67898d29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:48.109551 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a174f840-ef84-48a7-999e-228c67898d29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:48.117935 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a174f840-ef84-48a7-999e-228c67898d29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:48.119182 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a174f840-ef84-48a7-999e-228c67898d29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:48.120064 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a174f840-ef84-48a7-999e-228c67898d29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:48.121371 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a174f840-ef84-48a7-999e-228c67898d29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:48.122267 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a174f840-ef84-48a7-999e-228c67898d29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:48.123326 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a174f840-ef84-48a7-999e-228c67898d29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:48.340344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a174f840-ef84-48a7-999e-228c67898d29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:48.351804 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a174f840-ef84-48a7-999e-228c67898d29 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.246846 Oct 28 09:50:48.352588 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 387/793] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:48 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:48.379051 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6803595c-53fe-4b7f-b5f1-06086e26a7ee tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:48.380564 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6803595c-53fe-4b7f-b5f1-06086e26a7ee tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:48.393616 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6803595c-53fe-4b7f-b5f1-06086e26a7ee tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:48.394662 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6803595c-53fe-4b7f-b5f1-06086e26a7ee tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:48.395576 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6803595c-53fe-4b7f-b5f1-06086e26a7ee tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:48.396996 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6803595c-53fe-4b7f-b5f1-06086e26a7ee tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:48.398107 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6803595c-53fe-4b7f-b5f1-06086e26a7ee tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:48.399034 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6803595c-53fe-4b7f-b5f1-06086e26a7ee tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:48.423227 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:UPDATE=2,nova_cell1:SELECT=96,nova_cell1:INSERT=2 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:48.568990 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6803595c-53fe-4b7f-b5f1-06086e26a7ee tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:48.581304 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6803595c-53fe-4b7f-b5f1-06086e26a7ee tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1364 microversion: 2.1 time: 0.212097 Oct 28 09:50:48.581304 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 407/794] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:48 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1364 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:48.979845 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1bd47a79-8889-49c6-9d5f-91c773176bf0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:48.980824 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1bd47a79-8889-49c6-9d5f-91c773176bf0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:48.988456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1bd47a79-8889-49c6-9d5f-91c773176bf0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:48.989096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1bd47a79-8889-49c6-9d5f-91c773176bf0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:48.989626 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1bd47a79-8889-49c6-9d5f-91c773176bf0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:48.990580 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1bd47a79-8889-49c6-9d5f-91c773176bf0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:48.991539 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1bd47a79-8889-49c6-9d5f-91c773176bf0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:48.992126 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1bd47a79-8889-49c6-9d5f-91c773176bf0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:49.020030 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1bd47a79-8889-49c6-9d5f-91c773176bf0 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/ips" status: 200 len: 150 microversion: 2.1 time: 0.189125 Oct 28 09:50:49.020754 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 388/795] 10.222.0.25 () {58 vars in 1248 bytes} [Fri Oct 28 09:50:48 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/ips => generated 150 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:50:49.390578 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-bf555c07-7b48-439d-8b07-797ef59f8e0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:49.392018 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-bf555c07-7b48-439d-8b07-797ef59f8e0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:49.398671 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bf555c07-7b48-439d-8b07-797ef59f8e0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:49.399181 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bf555c07-7b48-439d-8b07-797ef59f8e0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:49.399701 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bf555c07-7b48-439d-8b07-797ef59f8e0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:49.400424 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bf555c07-7b48-439d-8b07-797ef59f8e0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:49.400988 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bf555c07-7b48-439d-8b07-797ef59f8e0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:49.401548 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-bf555c07-7b48-439d-8b07-797ef59f8e0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:49.573843 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-bf555c07-7b48-439d-8b07-797ef59f8e0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:49.588855 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-bf555c07-7b48-439d-8b07-797ef59f8e0f tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.211139 Oct 28 09:50:49.589896 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 408/796] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:49 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:50:49.615107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-40e37775-cca6-472b-a0d4-12a12a81f6b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:49.615845 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-40e37775-cca6-472b-a0d4-12a12a81f6b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:49.622838 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-40e37775-cca6-472b-a0d4-12a12a81f6b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:49.623465 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-40e37775-cca6-472b-a0d4-12a12a81f6b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:49.623967 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-40e37775-cca6-472b-a0d4-12a12a81f6b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:49.624843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-40e37775-cca6-472b-a0d4-12a12a81f6b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:49.626041 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-40e37775-cca6-472b-a0d4-12a12a81f6b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:49.626566 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-40e37775-cca6-472b-a0d4-12a12a81f6b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:49.798329 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-40e37775-cca6-472b-a0d4-12a12a81f6b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:49.806973 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-40e37775-cca6-472b-a0d4-12a12a81f6b6 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1364 microversion: 2.1 time: 0.195698 Oct 28 09:50:49.807567 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 389/797] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:49 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1364 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:50.463786 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-eddf1b41-8aa3-4287-88f3-3fb502ac2d1d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f1ad3fd5-08d1-4739-b5c4-7ed8e0f388bf"}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:50.466933 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-eddf1b41-8aa3-4287-88f3-3fb502ac2d1d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:50.474484 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eddf1b41-8aa3-4287-88f3-3fb502ac2d1d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:50.474979 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eddf1b41-8aa3-4287-88f3-3fb502ac2d1d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:50.475416 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eddf1b41-8aa3-4287-88f3-3fb502ac2d1d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:50.476127 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eddf1b41-8aa3-4287-88f3-3fb502ac2d1d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:50.476580 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eddf1b41-8aa3-4287-88f3-3fb502ac2d1d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:50.477001 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-eddf1b41-8aa3-4287-88f3-3fb502ac2d1d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:50.616550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-edab77c0-2a01-4c1a-ac60-947f14a30386 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:50.617722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-edab77c0-2a01-4c1a-ac60-947f14a30386 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:50.627383 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-edab77c0-2a01-4c1a-ac60-947f14a30386 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:50.627951 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-edab77c0-2a01-4c1a-ac60-947f14a30386 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:50.628429 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-edab77c0-2a01-4c1a-ac60-947f14a30386 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:50.629448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-edab77c0-2a01-4c1a-ac60-947f14a30386 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:50.630198 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-edab77c0-2a01-4c1a-ac60-947f14a30386 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:50.630687 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-edab77c0-2a01-4c1a-ac60-947f14a30386 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:50.638320 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:50.831944 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-edab77c0-2a01-4c1a-ac60-947f14a30386 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:50.845814 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-edab77c0-2a01-4c1a-ac60-947f14a30386 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.236346 Oct 28 09:50:50.846416 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 390/798] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:50 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:50.852580 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d611a030-b7fd-4428-b861-bd6677e58f7b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:50.853296 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d611a030-b7fd-4428-b861-bd6677e58f7b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:50.860998 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d611a030-b7fd-4428-b861-bd6677e58f7b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:50.861587 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d611a030-b7fd-4428-b861-bd6677e58f7b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:50.862095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d611a030-b7fd-4428-b861-bd6677e58f7b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:50.862930 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d611a030-b7fd-4428-b861-bd6677e58f7b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:50.863441 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d611a030-b7fd-4428-b861-bd6677e58f7b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:50.863905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d611a030-b7fd-4428-b861-bd6677e58f7b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:51.050862 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d611a030-b7fd-4428-b861-bd6677e58f7b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:51.061565 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d611a030-b7fd-4428-b861-bd6677e58f7b tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1364 microversion: 2.1 time: 0.213141 Oct 28 09:50:51.062341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 391/799] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:50 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1364 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:51.871909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c8a099c0-6b08-4857-b0a2-f81702d1d8c6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:51.873229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c8a099c0-6b08-4857-b0a2-f81702d1d8c6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:51.883347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8a099c0-6b08-4857-b0a2-f81702d1d8c6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:51.884201 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8a099c0-6b08-4857-b0a2-f81702d1d8c6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:51.884998 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8a099c0-6b08-4857-b0a2-f81702d1d8c6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:51.886345 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8a099c0-6b08-4857-b0a2-f81702d1d8c6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:51.887511 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8a099c0-6b08-4857-b0a2-f81702d1d8c6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:51.888427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8a099c0-6b08-4857-b0a2-f81702d1d8c6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:51.973958 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-eddf1b41-8aa3-4287-88f3-3fb502ac2d1d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "POST /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.514547 Oct 28 09:50:51.976079 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 409/800] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:50:50 2022] POST /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/os-volume_attachments => generated 194 bytes in 1518 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:50:52.101325 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-6088925d-8cc3-4e75-aabb-4983f349de54 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:52.102846 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.compute.api [None req-6088925d-8cc3-4e75-aabb-4983f349de54 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127029) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:52.117388 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6088925d-8cc3-4e75-aabb-4983f349de54 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:52.118513 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6088925d-8cc3-4e75-aabb-4983f349de54 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:52.119662 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6088925d-8cc3-4e75-aabb-4983f349de54 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:52.121218 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6088925d-8cc3-4e75-aabb-4983f349de54 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:52.122502 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6088925d-8cc3-4e75-aabb-4983f349de54 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:52.124113 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [None req-6088925d-8cc3-4e75-aabb-4983f349de54 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:52.302319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c8a099c0-6b08-4857-b0a2-f81702d1d8c6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:52.319555 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c8a099c0-6b08-4857-b0a2-f81702d1d8c6 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1523 microversion: 2.1 time: 0.452812 Oct 28 09:50:52.320526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 392/801] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:51 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1523 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:50:52.455082 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.policy [None req-6088925d-8cc3-4e75-aabb-4983f349de54 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127029) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:52.467714 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [None req-6088925d-8cc3-4e75-aabb-4983f349de54 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1550 microversion: 2.1 time: 0.378481 Oct 28 09:50:52.468837 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 410/802] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:52 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1550 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:52.509608 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d577d9c4-3d52-49d0-839e-017b6b4d896c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:52.510243 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d577d9c4-3d52-49d0-839e-017b6b4d896c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:52.520368 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d577d9c4-3d52-49d0-839e-017b6b4d896c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:52.521144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d577d9c4-3d52-49d0-839e-017b6b4d896c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:52.521689 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d577d9c4-3d52-49d0-839e-017b6b4d896c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:52.522588 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d577d9c4-3d52-49d0-839e-017b6b4d896c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:52.523181 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d577d9c4-3d52-49d0-839e-017b6b4d896c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:52.523899 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d577d9c4-3d52-49d0-839e-017b6b4d896c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:52.702996 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d577d9c4-3d52-49d0-839e-017b6b4d896c tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.198415 Oct 28 09:50:52.705606 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 393/803] 10.222.0.25 () {58 vars in 1275 bytes} [Fri Oct 28 09:50:52 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-interface => generated 284 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:50:52.792773 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-91b0c34b-ed1f-46f9-946e-dabbe56cb3bd req-d9078b71-0d9c-47d1-8d76-f5d4841786f7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b7d7553-45b6-4a10-868e-b653d211245d", "name": "network-vif-deleted", "tag": "bb15e303-a14b-4992-96a6-836072bb6949"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:52.815678 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-91b0c34b-ed1f-46f9-946e-dabbe56cb3bd req-d9078b71-0d9c-47d1-8d76-f5d4841786f7 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:52.816705 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-91b0c34b-ed1f-46f9-946e-dabbe56cb3bd req-d9078b71-0d9c-47d1-8d76-f5d4841786f7 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:52.817308 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-91b0c34b-ed1f-46f9-946e-dabbe56cb3bd req-d9078b71-0d9c-47d1-8d76-f5d4841786f7 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:52.832433 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-91b0c34b-ed1f-46f9-946e-dabbe56cb3bd req-d9078b71-0d9c-47d1-8d76-f5d4841786f7 service nova] Creating event network-vif-deleted:bb15e303-a14b-4992-96a6-836072bb6949 for instance 3b7d7553-45b6-4a10-868e-b653d211245d on hv-cinder-zed-3 Oct 28 09:50:52.838961 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-91b0c34b-ed1f-46f9-946e-dabbe56cb3bd req-d9078b71-0d9c-47d1-8d76-f5d4841786f7 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050342 Oct 28 09:50:52.840154 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 411/804] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:50:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:50:53.361375 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:53.362385 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:53.373169 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:53.374058 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:53.374883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:53.376187 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:53.377019 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:53.377792 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:53.429553 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:53.430027 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:53.430490 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:53.430962 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lazy-loading 'fault' on Instance uuid 3b7d7553-45b6-4a10-868e-b653d211245d {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:50:53.479657 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ed9a4c24cbfb4689b7f984219c7503a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '227ac6c4e642467ca3b9e0bf7181d5c9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:50:53.488499 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7be7141a-e95d-4795-8aaa-f981dfc53d73 tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 200 len: 1312 microversion: 2.1 time: 0.132491 Oct 28 09:50:53.489093 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 394/805] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:53 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 1312 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:50:54.110426 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [req-cae373d0-436b-4b32-bbf4-e5539edebe07 req-39e733e1-3bdb-49e0-b6ee-3fb5de40d310 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c28f1b03-788f-42f6-8577-9c81bffc41ae", "tag": "1fff934b-bb90-4e1c-b8bd-845cc947058d"}]} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:54.122686 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-cae373d0-436b-4b32-bbf4-e5539edebe07 req-39e733e1-3bdb-49e0-b6ee-3fb5de40d310 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:54.123168 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-cae373d0-436b-4b32-bbf4-e5539edebe07 req-39e733e1-3bdb-49e0-b6ee-3fb5de40d310 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:54.123803 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG oslo_concurrency.lockutils [req-cae373d0-436b-4b32-bbf4-e5539edebe07 req-39e733e1-3bdb-49e0-b6ee-3fb5de40d310 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127029) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:54.136769 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.compute.server_external_events [req-cae373d0-436b-4b32-bbf4-e5539edebe07 req-39e733e1-3bdb-49e0-b6ee-3fb5de40d310 service nova] Creating event network-changed:1fff934b-bb90-4e1c-b8bd-845cc947058d for instance c28f1b03-788f-42f6-8577-9c81bffc41ae on hv-cinder-zed-3 Oct 28 09:50:54.142701 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO nova.api.openstack.requestlog [req-cae373d0-436b-4b32-bbf4-e5539edebe07 req-39e733e1-3bdb-49e0-b6ee-3fb5de40d310 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.098152 Oct 28 09:50:54.143413 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: [pid: 127029|app: 0|req: 412/806] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:50:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:50:54.512880 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6e76c1a8-e1be-4562-a2f4-8420ed85449e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:50:54.513953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6e76c1a8-e1be-4562-a2f4-8420ed85449e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] [instance: 3b7d7553-45b6-4a10-868e-b653d211245d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:54.522747 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6e76c1a8-e1be-4562-a2f4-8420ed85449e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:54.523702 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6e76c1a8-e1be-4562-a2f4-8420ed85449e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:54.524432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6e76c1a8-e1be-4562-a2f4-8420ed85449e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:54.525446 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6e76c1a8-e1be-4562-a2f4-8420ed85449e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:54.526000 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6e76c1a8-e1be-4562-a2f4-8420ed85449e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:54.526733 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6e76c1a8-e1be-4562-a2f4-8420ed85449e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:54.542868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-6e76c1a8-e1be-4562-a2f4-8420ed85449e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] HTTP exception thrown: Instance 3b7d7553-45b6-4a10-868e-b653d211245d could not be found. Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6e76c1a8-e1be-4562-a2f4-8420ed85449e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Request method failure captured: Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d HTTP/1.1 Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sjq718s-6y9fBFjvzR8wDyF_4AlWYEmdP4icnuJ0s_5turdxYbjJNH08Q4u87xp47MpEwc_KXLuW4AwBVGzdBvRRVA1GVRVASenhMGF1IGhESh28OTGp6LsG6ZaKxtUJblyS6zcH0d96QlXz7dkrheK75c4DcQ-AxWdIWnFfDpBXE Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 227ac6c4e642467ca3b9e0bf7181d5c9 Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:50:54.544491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:50:54.547425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_227ac6c4e642467ca3b9e0bf7181d5c9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/227ac6c4e642467ca3b9e0bf7181d5c9"}]}] Oct 28 09:50:54.547425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:50:54.547425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 227ac6c4e642467ca3b9e0bf7181d5c9 Oct 28 09:50:54.547425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1296287768 Oct 28 09:50:54.547425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-ServerRescueNegativeTestJSON-1296287768-project Oct 28 09:50:54.547425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:50:54.547425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:50:54.547425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: ed9a4c24cbfb4689b7f984219c7503a4 Oct 28 09:50:54.547425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1296287768-project Oct 28 09:50:54.547425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:50:54.547425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 3b7d7553-45b6-4a10-868e-b653d211245d could not be found. Oct 28 09:50:54.547425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '3b7d7553-45b6-4a10-868e-b653d211245d'} Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3b7d7553-45b6-4a10-868e-b653d211245d could not be found. Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:50:54.548883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3b7d7553-45b6-4a10-868e-b653d211245d could not be found. Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3b7d7553-45b6-4a10-868e-b653d211245d could not be found. Oct 28 09:50:54.553091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:50:54.558745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6e76c1a8-e1be-4562-a2f4-8420ed85449e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] Returning 404 to user: Instance 3b7d7553-45b6-4a10-868e-b653d211245d could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:50:54.558745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6e76c1a8-e1be-4562-a2f4-8420ed85449e tempest-ServerRescueNegativeTestJSON-1296287768 tempest-ServerRescueNegativeTestJSON-1296287768-project] 10.222.0.25 "GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d" status: 404 len: 111 microversion: 2.1 time: 0.037422 Oct 28 09:50:54.558745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 395/807] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:50:54 2022] GET /compute/v2.1/servers/3b7d7553-45b6-4a10-868e-b653d211245d => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:50:59.131464 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.api.openstack.wsgi [None req-c814cc22-3d22-43ef-82ad-ad93ff9a10b8 tempest-TestVolumeBootPattern-1984064024 tempest-TestVolumeBootPattern-1984064024-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-407395012", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-196867844", "security_groups": [{"name": "tempest-secgroup-smoke-910857212"}], "block_device_mapping_v2": [{"uuid": "67512c1d-ba0e-4904-a609-e29ac4ce5a1e", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "8b7ca88a-ecf5-4f0a-be39-1cb8545558c8"}]}} {{(pid=127029) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:59.314868 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG nova.network.neutron [None req-c814cc22-3d22-43ef-82ad-ad93ff9a10b8 tempest-TestVolumeBootPattern-1984064024 tempest-TestVolumeBootPattern-1984064024-project] validate_networks() for [('8b7ca88a-ecf5-4f0a-be39-1cb8545558c8', None, None, None, None, None)] {{(pid=127029) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:50:59.909911 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-59a559be-6d70-4e43-ab04-9a0806b105d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f1ad3fd5-08d1-4739-b5c4-7ed8e0f388bf"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:50:59.913121 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-59a559be-6d70-4e43-ab04-9a0806b105d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:50:59.922808 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a559be-6d70-4e43-ab04-9a0806b105d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:59.923348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a559be-6d70-4e43-ab04-9a0806b105d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:59.923792 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a559be-6d70-4e43-ab04-9a0806b105d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:50:59.924509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a559be-6d70-4e43-ab04-9a0806b105d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:50:59.924958 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a559be-6d70-4e43-ab04-9a0806b105d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:50:59.925378 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59a559be-6d70-4e43-ab04-9a0806b105d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:00.020483 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-59a559be-6d70-4e43-ab04-9a0806b105d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Invalid volume: volume f1ad3fd5-08d1-4739-b5c4-7ed8e0f388bf is already attached to instances: 576e3364-9c2e-4360-873a-7a9ded31e8e3 Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-59a559be-6d70-4e43-ab04-9a0806b105d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: POST /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/os-volume_attachments HTTP/1.1 Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 74 Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:51:00.022119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:51:00.024783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:51:00.024783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:51:00.024783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:51:00.024783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:51:00.024783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:51:00.024783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:51:00.024783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:51:00.024783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:51:00.024783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:51:00.024783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Oct 28 09:51:00.024783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {"volumeAttachment": {"volumeId": "f1ad3fd5-08d1-4739-b5c4-7ed8e0f388bf"}} Oct 28 09:51:00.024783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Invalid volume: volume f1ad3fd5-08d1-4739-b5c4-7ed8e0f388bf is already attached to instances: 576e3364-9c2e-4360-873a-7a9ded31e8e3 Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'server_id': '576e3364-9c2e-4360-873a-7a9ded31e8e3', 'body': {'volumeAttachment': {'volumeId': 'f1ad3fd5-08d1-4739-b5c4-7ed8e0f388bf'}}} Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume f1ad3fd5-08d1-4739-b5c4-7ed8e0f388bf is already attached to instances: 576e3364-9c2e-4360-873a-7a9ded31e8e3 Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.025600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume f1ad3fd5-08d1-4739-b5c4-7ed8e0f388bf is already attached to instances: 576e3364-9c2e-4360-873a-7a9ded31e8e3 Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.027841 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.030683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.030683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:51:00.030683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:51:00.030683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume f1ad3fd5-08d1-4739-b5c4-7ed8e0f388bf is already attached to instances: 576e3364-9c2e-4360-873a-7a9ded31e8e3 Oct 28 09:51:00.030683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.030683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-59a559be-6d70-4e43-ab04-9a0806b105d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 400 to user: Invalid volume: volume f1ad3fd5-08d1-4739-b5c4-7ed8e0f388bf is already attached to instances: 576e3364-9c2e-4360-873a-7a9ded31e8e3 {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:51:00.030683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-59a559be-6d70-4e43-ab04-9a0806b105d4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "POST /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.118521 Oct 28 09:51:00.030683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 396/808] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:50:59 2022] POST /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/os-volume_attachments => generated 174 bytes in 120 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 28 09:51:00.111449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-49b680fa-ae84-488c-8ddf-1504a74212c6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:00.112037 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-49b680fa-ae84-488c-8ddf-1504a74212c6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:00.123067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-49b680fa-ae84-488c-8ddf-1504a74212c6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:00.123905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-49b680fa-ae84-488c-8ddf-1504a74212c6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:00.124543 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-49b680fa-ae84-488c-8ddf-1504a74212c6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:00.125709 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-49b680fa-ae84-488c-8ddf-1504a74212c6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:00.126664 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-49b680fa-ae84-488c-8ddf-1504a74212c6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:00.127638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-49b680fa-ae84-488c-8ddf-1504a74212c6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:00.240177 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-49b680fa-ae84-488c-8ddf-1504a74212c6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lazy-loading 'services' on Instance uuid 576e3364-9c2e-4360-873a-7a9ded31e8e3 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:51:00.344667 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-49b680fa-ae84-488c-8ddf-1504a74212c6 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/os-volume_attachments/f1ad3fd5-08d1-4739-b5c4-7ed8e0f388bf" status: 202 len: 0 microversion: 2.1 time: 0.237209 Oct 28 09:51:00.345758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 397/809] 10.222.0.25 () {58 vars in 1416 bytes} [Fri Oct 28 09:51:00 2022] DELETE /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/os-volume_attachments/f1ad3fd5-08d1-4739-b5c4-7ed8e0f388bf => generated 0 bytes in 241 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 09:51:00.529942 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f22fc459-64ad-49db-b0a0-21d11157d586 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Action: 'create', calling method: .version_select at 0x7f271a3958b0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1657976266", "metadata": {"Type": "work"}}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f22fc459-64ad-49db-b0a0-21d11157d586 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Request method failure captured: Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 122 Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6YEhLju3d_qys8UvXl02vykXDRPHAr5hwH882M6xgdxVq9A-_waCJTRH0NYVD1jZJ24OIJT_Ym8MbqkteGhoY8QTMifxvfqN5TXq8ttK7oPyO6QvExiMgm4_-mFL4G02ETeEw-A4DUVRGQtrgHheSo_xMYMG1KnHnDSPca5W0WUHNw Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 25e4cd851ba947ed96286977236953a6 Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:51:00.541806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:51:00.544348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/25e4cd851ba947ed96286977236953a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/25e4cd851ba947ed96286977236953a6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_25e4cd851ba947ed96286977236953a6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/25e4cd851ba947ed96286977236953a6"}]}] Oct 28 09:51:00.544348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.544348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 25e4cd851ba947ed96286977236953a6 Oct 28 09:51:00.544348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.544348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-VolumesNegativeTest-831417308-project Oct 28 09:51:00.544348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:51:00.544348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:51:00.544348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 838463b8eca244218e50fa343d4b5545 Oct 28 09:51:00.544348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-VolumesNegativeTest-831417308-project Oct 28 09:51:00.544348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Oct 28 09:51:00.544348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1657976266", "metadata": {"Type": "work"}}} Oct 28 09:51:00.544348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: Oct 28 09:51:00.544348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1657976266', 'metadata': {'Type': 'work'}}}} Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise error Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi '#$%' Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:51:00.546442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 28 09:51:00.550235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 28 09:51:00.553362 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 28 09:51:00.553362 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.553362 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f22fc459-64ad-49db-b0a0-21d11157d586 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:51:00.553362 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f22fc459-64ad-49db-b0a0-21d11157d586 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.196334 Oct 28 09:51:00.553362 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 398/810] 10.222.0.25 () {60 vars in 1156 bytes} [Fri Oct 28 09:51:00 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 198 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 28 09:51:00.553362 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b99935cd-cfe3-477f-b152-4d1435e86d8e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:00.556579 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b99935cd-cfe3-477f-b152-4d1435e86d8e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:00.565855 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b99935cd-cfe3-477f-b152-4d1435e86d8e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:00.566464 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b99935cd-cfe3-477f-b152-4d1435e86d8e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:00.566908 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b99935cd-cfe3-477f-b152-4d1435e86d8e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:00.567665 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b99935cd-cfe3-477f-b152-4d1435e86d8e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:00.568159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b99935cd-cfe3-477f-b152-4d1435e86d8e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:00.568620 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b99935cd-cfe3-477f-b152-4d1435e86d8e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:00.605233 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b99935cd-cfe3-477f-b152-4d1435e86d8e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059138 Oct 28 09:51:00.605966 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 399/811] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:51:00 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:51:00.615019 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f89d462f-9c5c-4560-99e2-4780814d237e tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Action: 'create', calling method: .version_select at 0x7f271dd5ee50>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-400225428", "metadata": {"Type": "work"}}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f89d462f-9c5c-4560-99e2-4780814d237e tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Request method failure captured: Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 119 Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6YEhLju3d_qys8UvXl02vykXDRPHAr5hwH882M6xgdxVq9A-_waCJTRH0NYVD1jZJ24OIJT_Ym8MbqkteGhoY8QTMifxvfqN5TXq8ttK7oPyO6QvExiMgm4_-mFL4G02ETeEw-A4DUVRGQtrgHheSo_xMYMG1KnHnDSPca5W0WUHNw Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 25e4cd851ba947ed96286977236953a6 Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:51:00.619073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:51:00.621781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/25e4cd851ba947ed96286977236953a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/25e4cd851ba947ed96286977236953a6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_25e4cd851ba947ed96286977236953a6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/25e4cd851ba947ed96286977236953a6"}]}] Oct 28 09:51:00.621781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.621781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 25e4cd851ba947ed96286977236953a6 Oct 28 09:51:00.621781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.621781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-VolumesNegativeTest-831417308-project Oct 28 09:51:00.621781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:51:00.621781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:51:00.621781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 838463b8eca244218e50fa343d4b5545 Oct 28 09:51:00.621781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-VolumesNegativeTest-831417308-project Oct 28 09:51:00.621781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Oct 28 09:51:00.621781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-400225428", "metadata": {"Type": "work"}}} Oct 28 09:51:00.621781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: Oct 28 09:51:00.621781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-400225428', 'metadata': {'Type': 'work'}}}} Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise error Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi '0' Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:51:00.623125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 28 09:51:00.625819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 28 09:51:00.627914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 28 09:51:00.627914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.627914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f89d462f-9c5c-4560-99e2-4780814d237e tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:51:00.627914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f89d462f-9c5c-4560-99e2-4780814d237e tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.013886 Oct 28 09:51:00.627914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 400/812] 10.222.0.25 () {60 vars in 1156 bytes} [Fri Oct 28 09:51:00 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 16 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 28 09:51:00.635566 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-916b2fbc-7ec4-4690-98a8-e8afe51ca76e tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Action: 'create', calling method: .version_select at 0x7f271a395820>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1534869769", "metadata": {"Type": "work"}}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-916b2fbc-7ec4-4690-98a8-e8afe51ca76e tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Request method failure captured: Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 119 Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6YEhLju3d_qys8UvXl02vykXDRPHAr5hwH882M6xgdxVq9A-_waCJTRH0NYVD1jZJ24OIJT_Ym8MbqkteGhoY8QTMifxvfqN5TXq8ttK7oPyO6QvExiMgm4_-mFL4G02ETeEw-A4DUVRGQtrgHheSo_xMYMG1KnHnDSPca5W0WUHNw Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 25e4cd851ba947ed96286977236953a6 Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:51:00.639496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:51:00.642042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/25e4cd851ba947ed96286977236953a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/25e4cd851ba947ed96286977236953a6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_25e4cd851ba947ed96286977236953a6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/25e4cd851ba947ed96286977236953a6"}]}] Oct 28 09:51:00.642042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.642042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 25e4cd851ba947ed96286977236953a6 Oct 28 09:51:00.642042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.642042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-VolumesNegativeTest-831417308-project Oct 28 09:51:00.642042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:51:00.642042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:51:00.642042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 838463b8eca244218e50fa343d4b5545 Oct 28 09:51:00.642042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-VolumesNegativeTest-831417308-project Oct 28 09:51:00.642042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Oct 28 09:51:00.642042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1534869769", "metadata": {"Type": "work"}}} Oct 28 09:51:00.642042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: Oct 28 09:51:00.642042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1534869769', 'metadata': {'Type': 'work'}}}} Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise error Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi '' Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:51:00.643771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 28 09:51:00.646616 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 28 09:51:00.649491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 28 09:51:00.649491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.649491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-916b2fbc-7ec4-4690-98a8-e8afe51ca76e tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:51:00.649491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-916b2fbc-7ec4-4690-98a8-e8afe51ca76e tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.009331 Oct 28 09:51:00.649491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 401/813] 10.222.0.25 () {60 vars in 1156 bytes} [Fri Oct 28 09:51:00 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 28 09:51:00.654843 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-dcadfcd9-38e4-46b0-ad57-c6e696deab58 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Calling method '.version_select at 0x7f271a395d30>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:00.687185 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-dcadfcd9-38e4-46b0-ad57-c6e696deab58 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] HTTP exception thrown: Volume tempest-invalid-30209775 could not be found. Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-dcadfcd9-38e4-46b0-ad57-c6e696deab58 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Request method failure captured: Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-30209775 HTTP/1.1 Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6YEhLju3d_qys8UvXl02vykXDRPHAr5hwH882M6xgdxVq9A-_waCJTRH0NYVD1jZJ24OIJT_Ym8MbqkteGhoY8QTMifxvfqN5TXq8ttK7oPyO6QvExiMgm4_-mFL4G02ETeEw-A4DUVRGQtrgHheSo_xMYMG1KnHnDSPca5W0WUHNw Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 25e4cd851ba947ed96286977236953a6 Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:51:00.689691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:51:00.692031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/25e4cd851ba947ed96286977236953a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/25e4cd851ba947ed96286977236953a6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_25e4cd851ba947ed96286977236953a6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/25e4cd851ba947ed96286977236953a6"}]}] Oct 28 09:51:00.692031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.692031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 25e4cd851ba947ed96286977236953a6 Oct 28 09:51:00.692031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.692031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-VolumesNegativeTest-831417308-project Oct 28 09:51:00.692031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:51:00.692031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:51:00.692031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 838463b8eca244218e50fa343d4b5545 Oct 28 09:51:00.692031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-VolumesNegativeTest-831417308-project Oct 28 09:51:00.692031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: Oct 28 09:51:00.692031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Volume tempest-invalid-30209775 could not be found. Oct 28 09:51:00.692031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'tempest-invalid-30209775'} Oct 28 09:51:00.692031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-30209775 could not be found. (HTTP 404) (Request-ID: req-afbd2128-ef1f-46f0-971d-095801df3e4c) Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.693508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 28 09:51:00.695874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-30209775 could not be found. Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:51:00.698316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-30209775 could not be found. Oct 28 09:51:00.700974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.700974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.700974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.700974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.700974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.700974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.700974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:51:00.700974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:51:00.700974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-30209775 could not be found. Oct 28 09:51:00.700974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.700974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-dcadfcd9-38e4-46b0-ad57-c6e696deab58 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Returning 404 to user: Volume tempest-invalid-30209775 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:51:00.700974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-dcadfcd9-38e4-46b0-ad57-c6e696deab58 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/tempest-invalid-30209775" status: 404 len: 97 microversion: 2.1 time: 0.041228 Oct 28 09:51:00.700974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 402/814] 10.222.0.25 () {58 vars in 1212 bytes} [Fri Oct 28 09:51:00 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-30209775 => generated 97 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 28 09:51:00.704596 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-26951e6f-967d-45b2-865b-aaf0e0df64b9 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.005997 Oct 28 09:51:00.705178 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 403/815] 10.222.0.25 () {58 vars in 1140 bytes} [Fri Oct 28 09:51:00 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 8 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Oct 28 09:51:00.715280 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3bfc0e4e-9acf-4b64-be69-abedea323e79 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003832 Oct 28 09:51:00.715997 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 404/816] 10.222.0.25 () {58 vars in 1137 bytes} [Fri Oct 28 09:51:00 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Oct 28 09:51:00.726692 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-00927fdc-4993-4d8d-b197-472676bab120 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Calling method '.version_select at 0x7f271dd2b8b0>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:00.773419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-00927fdc-4993-4d8d-b197-472676bab120 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] HTTP exception thrown: Volume 43c96b9b-723c-47bc-9afc-6f8b240a4a89 could not be found. Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-00927fdc-4993-4d8d-b197-472676bab120 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Request method failure captured: Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: DELETE /compute/v2.1/os-volumes/43c96b9b-723c-47bc-9afc-6f8b240a4a89 HTTP/1.1 Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6YEhLju3d_qys8UvXl02vykXDRPHAr5hwH882M6xgdxVq9A-_waCJTRH0NYVD1jZJ24OIJT_Ym8MbqkteGhoY8QTMifxvfqN5TXq8ttK7oPyO6QvExiMgm4_-mFL4G02ETeEw-A4DUVRGQtrgHheSo_xMYMG1KnHnDSPca5W0WUHNw Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 25e4cd851ba947ed96286977236953a6 Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:51:00.776036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:51:00.778527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/25e4cd851ba947ed96286977236953a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/25e4cd851ba947ed96286977236953a6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_25e4cd851ba947ed96286977236953a6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/25e4cd851ba947ed96286977236953a6"}]}] Oct 28 09:51:00.778527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.778527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 25e4cd851ba947ed96286977236953a6 Oct 28 09:51:00.778527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.778527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-VolumesNegativeTest-831417308-project Oct 28 09:51:00.778527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:51:00.778527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:51:00.778527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 838463b8eca244218e50fa343d4b5545 Oct 28 09:51:00.778527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-VolumesNegativeTest-831417308-project Oct 28 09:51:00.778527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: Oct 28 09:51:00.778527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Volume 43c96b9b-723c-47bc-9afc-6f8b240a4a89 could not be found. Oct 28 09:51:00.778527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '43c96b9b-723c-47bc-9afc-6f8b240a4a89'} Oct 28 09:51:00.778527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 43c96b9b-723c-47bc-9afc-6f8b240a4a89 could not be found. (HTTP 404) (Request-ID: req-ff479d0d-6873-442e-8a71-0fec4575aeb4) Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.779989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 28 09:51:00.783204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 43c96b9b-723c-47bc-9afc-6f8b240a4a89 could not be found. Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 28 09:51:00.785347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 43c96b9b-723c-47bc-9afc-6f8b240a4a89 could not be found. Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 43c96b9b-723c-47bc-9afc-6f8b240a4a89 could not be found. Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-00927fdc-4993-4d8d-b197-472676bab120 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Returning 404 to user: Volume 43c96b9b-723c-47bc-9afc-6f8b240a4a89 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-00927fdc-4993-4d8d-b197-472676bab120 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/43c96b9b-723c-47bc-9afc-6f8b240a4a89" status: 404 len: 109 microversion: 2.1 time: 0.054854 Oct 28 09:51:00.787529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 405/817] 10.222.0.25 () {58 vars in 1248 bytes} [Fri Oct 28 09:51:00 2022] DELETE /compute/v2.1/os-volumes/43c96b9b-723c-47bc-9afc-6f8b240a4a89 => generated 109 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 09:51:00.792506 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e3391fe2-4252-491d-b014-16b0d7177c79 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Calling method '.version_select at 0x7f271dd3ca60>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:00.831460 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-e3391fe2-4252-491d-b014-16b0d7177c79 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] HTTP exception thrown: Volume f2decf10-a5fb-42f6-bb3e-fb828082a7e9 could not be found. Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e3391fe2-4252-491d-b014-16b0d7177c79 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Request method failure captured: Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/os-volumes/f2decf10-a5fb-42f6-bb3e-fb828082a7e9 HTTP/1.1 Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6YEhLju3d_qys8UvXl02vykXDRPHAr5hwH882M6xgdxVq9A-_waCJTRH0NYVD1jZJ24OIJT_Ym8MbqkteGhoY8QTMifxvfqN5TXq8ttK7oPyO6QvExiMgm4_-mFL4G02ETeEw-A4DUVRGQtrgHheSo_xMYMG1KnHnDSPca5W0WUHNw Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 25e4cd851ba947ed96286977236953a6 Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:51:00.833405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:51:00.834981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/25e4cd851ba947ed96286977236953a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/25e4cd851ba947ed96286977236953a6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_25e4cd851ba947ed96286977236953a6"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/25e4cd851ba947ed96286977236953a6"}]}] Oct 28 09:51:00.834981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.834981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 25e4cd851ba947ed96286977236953a6 Oct 28 09:51:00.834981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-VolumesNegativeTest-831417308 Oct 28 09:51:00.834981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-VolumesNegativeTest-831417308-project Oct 28 09:51:00.834981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:51:00.834981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:51:00.834981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 838463b8eca244218e50fa343d4b5545 Oct 28 09:51:00.834981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-VolumesNegativeTest-831417308-project Oct 28 09:51:00.834981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: Oct 28 09:51:00.834981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Volume f2decf10-a5fb-42f6-bb3e-fb828082a7e9 could not be found. Oct 28 09:51:00.834981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'f2decf10-a5fb-42f6-bb3e-fb828082a7e9'} Oct 28 09:51:00.834981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume f2decf10-a5fb-42f6-bb3e-fb828082a7e9 could not be found. (HTTP 404) (Request-ID: req-cc2ea9ab-1d02-4a4a-8cf9-d03027406fe4) Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.835768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 09:51:00.837044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume f2decf10-a5fb-42f6-bb3e-fb828082a7e9 could not be found. Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 28 09:51:00.838847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume f2decf10-a5fb-42f6-bb3e-fb828082a7e9 could not be found. Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume f2decf10-a5fb-42f6-bb3e-fb828082a7e9 could not be found. Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e3391fe2-4252-491d-b014-16b0d7177c79 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] Returning 404 to user: Volume f2decf10-a5fb-42f6-bb3e-fb828082a7e9 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e3391fe2-4252-491d-b014-16b0d7177c79 tempest-VolumesNegativeTest-831417308 tempest-VolumesNegativeTest-831417308-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/f2decf10-a5fb-42f6-bb3e-fb828082a7e9" status: 404 len: 109 microversion: 2.1 time: 0.050864 Oct 28 09:51:00.840586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 406/818] 10.222.0.25 () {58 vars in 1245 bytes} [Fri Oct 28 09:51:00 2022] GET /compute/v2.1/os-volumes/f2decf10-a5fb-42f6-bb3e-fb828082a7e9 => generated 109 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 09:51:01.629472 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c9e57bcc-854b-4afa-8830-32a65174a328 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:01.635984 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c9e57bcc-854b-4afa-8830-32a65174a328 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:01.649107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9e57bcc-854b-4afa-8830-32a65174a328 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:01.649788 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9e57bcc-854b-4afa-8830-32a65174a328 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:01.650376 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9e57bcc-854b-4afa-8830-32a65174a328 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:01.652124 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9e57bcc-854b-4afa-8830-32a65174a328 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:01.653684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9e57bcc-854b-4afa-8830-32a65174a328 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:01.654482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9e57bcc-854b-4afa-8830-32a65174a328 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:01.695214 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c9e57bcc-854b-4afa-8830-32a65174a328 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.075532 Oct 28 09:51:01.696046 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 407/819] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:51:01 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3/os-volume_attachments => generated 25 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 09:51:01.780435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5df6b76b-a0cf-49d7-946c-d18fd064ac01 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:01.781223 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5df6b76b-a0cf-49d7-946c-d18fd064ac01 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:01.794764 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5df6b76b-a0cf-49d7-946c-d18fd064ac01 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:01.795748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5df6b76b-a0cf-49d7-946c-d18fd064ac01 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:01.797831 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5df6b76b-a0cf-49d7-946c-d18fd064ac01 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:01.799147 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5df6b76b-a0cf-49d7-946c-d18fd064ac01 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:01.799974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5df6b76b-a0cf-49d7-946c-d18fd064ac01 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:01.800725 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5df6b76b-a0cf-49d7-946c-d18fd064ac01 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:01.835850 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5df6b76b-a0cf-49d7-946c-d18fd064ac01 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Going to try to terminate instance {{(pid=127030) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:51:01.948060 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5df6b76b-a0cf-49d7-946c-d18fd064ac01 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 204 len: 0 microversion: 2.1 time: 0.171635 Oct 28 09:51:01.949335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 408/820] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:51:01 2022] DELETE /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 28 09:51:01.962501 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-721129bb-b82a-4601-b823-7b26b0077c94 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:01.963602 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-721129bb-b82a-4601-b823-7b26b0077c94 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:01.971459 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-721129bb-b82a-4601-b823-7b26b0077c94 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:01.972240 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-721129bb-b82a-4601-b823-7b26b0077c94 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:01.973006 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-721129bb-b82a-4601-b823-7b26b0077c94 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:01.973911 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-721129bb-b82a-4601-b823-7b26b0077c94 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:01.974575 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-721129bb-b82a-4601-b823-7b26b0077c94 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:01.975218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-721129bb-b82a-4601-b823-7b26b0077c94 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:02.140222 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-721129bb-b82a-4601-b823-7b26b0077c94 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:02.153948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-721129bb-b82a-4601-b823-7b26b0077c94 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1682 microversion: 2.1 time: 0.197706 Oct 28 09:51:02.154999 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 409/821] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:01 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1682 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:51:03.180085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-eaaaae2c-93c2-49b9-85f4-45624194b302 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:03.181094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-eaaaae2c-93c2-49b9-85f4-45624194b302 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:03.188098 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eaaaae2c-93c2-49b9-85f4-45624194b302 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:03.188673 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eaaaae2c-93c2-49b9-85f4-45624194b302 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:03.189110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eaaaae2c-93c2-49b9-85f4-45624194b302 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:03.189822 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eaaaae2c-93c2-49b9-85f4-45624194b302 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:03.190277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eaaaae2c-93c2-49b9-85f4-45624194b302 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:03.190752 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eaaaae2c-93c2-49b9-85f4-45624194b302 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:03.387511 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-eaaaae2c-93c2-49b9-85f4-45624194b302 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:03.397396 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-eaaaae2c-93c2-49b9-85f4-45624194b302 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1682 microversion: 2.1 time: 0.221145 Oct 28 09:51:03.398270 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 410/822] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:03 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1682 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:51:04.129900 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_cell1:SELECT=29,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:04.425201 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d4cee14b-4668-44a1-9378-0b6a421ac92e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:04.426096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d4cee14b-4668-44a1-9378-0b6a421ac92e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:04.434686 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d4cee14b-4668-44a1-9378-0b6a421ac92e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:04.435507 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d4cee14b-4668-44a1-9378-0b6a421ac92e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:04.436258 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d4cee14b-4668-44a1-9378-0b6a421ac92e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:04.437161 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d4cee14b-4668-44a1-9378-0b6a421ac92e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:04.438192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d4cee14b-4668-44a1-9378-0b6a421ac92e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:04.438717 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d4cee14b-4668-44a1-9378-0b6a421ac92e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:04.759157 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d4cee14b-4668-44a1-9378-0b6a421ac92e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:04.779374 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d4cee14b-4668-44a1-9378-0b6a421ac92e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1682 microversion: 2.1 time: 0.359373 Oct 28 09:51:04.794455 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 411/823] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:04 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1682 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:51:05.238937 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-026980bd-9178-4cbf-93fc-e4e77002e4bb req-5f9c6bb2-a257-4950-bc43-b8c6e963a0f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "576e3364-9c2e-4360-873a-7a9ded31e8e3", "name": "network-vif-deleted", "tag": "b0640bfe-73f9-4cda-a3ed-720c31e1391d"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:05.256084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-026980bd-9178-4cbf-93fc-e4e77002e4bb req-5f9c6bb2-a257-4950-bc43-b8c6e963a0f4 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:05.262480 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-026980bd-9178-4cbf-93fc-e4e77002e4bb req-5f9c6bb2-a257-4950-bc43-b8c6e963a0f4 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:05.262809 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-026980bd-9178-4cbf-93fc-e4e77002e4bb req-5f9c6bb2-a257-4950-bc43-b8c6e963a0f4 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:05.283283 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-026980bd-9178-4cbf-93fc-e4e77002e4bb req-5f9c6bb2-a257-4950-bc43-b8c6e963a0f4 service nova] Creating event network-vif-deleted:b0640bfe-73f9-4cda-a3ed-720c31e1391d for instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 on hv-cinder-zed-3 Oct 28 09:51:05.296646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-026980bd-9178-4cbf-93fc-e4e77002e4bb req-5f9c6bb2-a257-4950-bc43-b8c6e963a0f4 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063978 Oct 28 09:51:05.298645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 412/824] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:51:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:51:05.826432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:05.829291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:05.840815 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:05.841626 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:05.842351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:05.843928 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:05.844580 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:05.845525 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:05.889831 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:05.890384 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:05.891657 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:05.892366 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lazy-loading 'fault' on Instance uuid 576e3364-9c2e-4360-873a-7a9ded31e8e3 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:51:05.895049 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:05.967626 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:05.986303 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0a21b38e-1e3f-447c-b709-e3ad537ccc98 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 200 len: 1331 microversion: 2.1 time: 0.168059 Oct 28 09:51:05.987965 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 413/825] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:05 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 1331 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:51:07.025462 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b9c99a58-087c-42d6-87fd-095444d11c09 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:07.026334 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b9c99a58-087c-42d6-87fd-095444d11c09 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:07.033316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b9c99a58-087c-42d6-87fd-095444d11c09 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:07.034129 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b9c99a58-087c-42d6-87fd-095444d11c09 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:07.034658 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b9c99a58-087c-42d6-87fd-095444d11c09 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:07.035489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b9c99a58-087c-42d6-87fd-095444d11c09 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:07.035936 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b9c99a58-087c-42d6-87fd-095444d11c09 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:07.036350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b9c99a58-087c-42d6-87fd-095444d11c09 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:07.051900 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-b9c99a58-087c-42d6-87fd-095444d11c09 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b9c99a58-087c-42d6-87fd-095444d11c09 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 HTTP/1.1 Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:51:07.053574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:51:07.056737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:51:07.056737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:51:07.056737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:51:07.056737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:51:07.056737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:51:07.056737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:51:07.056737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:51:07.056737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:51:07.056737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:51:07.056737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:51:07.056737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:51:07.056737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '576e3364-9c2e-4360-873a-7a9ded31e8e3'} Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:51:07.058488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:51:07.061473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:51:07.063985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b9c99a58-087c-42d6-87fd-095444d11c09 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 404 to user: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:51:07.063985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b9c99a58-087c-42d6-87fd-095444d11c09 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 404 len: 111 microversion: 2.1 time: 0.033640 Oct 28 09:51:07.063985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 414/826] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:07 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 09:51:07.064451 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2a321c70-443c-4eee-a6b9-da6a2fd95bfe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '.version_select at 0x7f271a322b80>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:07.096921 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2a321c70-443c-4eee-a6b9-da6a2fd95bfe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2064819975" status: 202 len: 0 microversion: 2.1 time: 0.036210 Oct 28 09:51:07.097879 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 415/827] 10.222.0.25 () {58 vars in 1221 bytes} [Fri Oct 28 09:51:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2064819975 => generated 0 bytes in 38 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 09:51:07.967148 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f1d48924-4870-48bf-8cc1-ffb93f51a1a4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Action: 'create', calling method: .version_select at 0x7f271a3423a0>, body: {"keypair": {"name": "tempest-keypair-530370620", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHiO68o1daY4o3HUfcG6X4F5tWwYqvrMSlJ3rT6+ppZVqNtHkfDjpcDcouDNhDeePyufqI3T0C6jQDweL3VIc291+loDJa4uv7FjY2ALQzAuE92GXIkbJLPDzQ30E5lfhw=="}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:07.988132 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-f1d48924-4870-48bf-8cc1-ffb93f51a1a4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for project 4f351d016ace4e369be852bb7aa916fe. Resources: {'key_pairs'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:51:07.992772 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-f1d48924-4870-48bf-8cc1-ffb93f51a1a4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for user e65d298cca0f478eba502edf5bd13954 and project 4f351d016ace4e369be852bb7aa916fe. Resources: {'key_pairs'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:51:08.020985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f1d48924-4870-48bf-8cc1-ffb93f51a1a4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.060269 Oct 28 09:51:08.022345 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 416/828] 10.222.0.25 () {60 vars in 1159 bytes} [Fri Oct 28 09:51:07 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:51:08.845830 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6e96d26c-4466-419d-ad62-fc87d8c79239 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:08.847135 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6e96d26c-4466-419d-ad62-fc87d8c79239 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:08.854181 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6e96d26c-4466-419d-ad62-fc87d8c79239 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:08.855270 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6e96d26c-4466-419d-ad62-fc87d8c79239 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:08.855717 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6e96d26c-4466-419d-ad62-fc87d8c79239 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:08.856455 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6e96d26c-4466-419d-ad62-fc87d8c79239 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:08.856908 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6e96d26c-4466-419d-ad62-fc87d8c79239 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:08.857322 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6e96d26c-4466-419d-ad62-fc87d8c79239 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:08.893905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6e96d26c-4466-419d-ad62-fc87d8c79239 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/ips" status: 200 len: 152 microversion: 2.20 time: 0.051956 Oct 28 09:51:08.894951 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 417/829] 10.222.0.25 () {60 vars in 1289 bytes} [Fri Oct 28 09:51:08 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/ips => generated 152 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 28 09:51:09.148795 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: DEBUG dbcounter [-] [127029] Writing DB stats nova_api:SELECT=57,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=127029) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:10.176075 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1911323353", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8cae7e07-cd54-4f98-96b4-a27c37dcb83a"}], "security_groups": [{"name": "tempest-securitygroup--675896182"}], "key_name": "tempest-keypair-530370620", "user_data": "}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:10.308933 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.network.neutron [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] validate_networks() for [('8cae7e07-cd54-4f98-96b4-a27c37dcb83a', None, None, None, None, None)] {{(pid=127030) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:51:10.852848 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Flavor limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:51:10.853631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Image limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:51:10.855068 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Flavor pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:51:10.856313 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Image pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:51:10.857155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:51:10.875329 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:10.876119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:10.876838 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:10.878082 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:10.878964 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:10.879665 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:10.908263 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for project 4f351d016ace4e369be852bb7aa916fe. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:51:10.914291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Getting quotas for user e65d298cca0f478eba502edf5bd13954 and project 4f351d016ace4e369be852bb7aa916fe. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:51:10.937523 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Going to run 1 instances... {{(pid=127030) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:51:10.941942 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127030) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:51:11.014445 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4a22cda5-23d2-40a2-bf23-6c70ce15440d tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.843652 Oct 28 09:51:11.016154 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 418/830] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:51:10 2022] POST /compute/v2.1/servers => generated 401 bytes in 846 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 28 09:51:11.026670 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ec1c6780-5bb1-4808-b89a-c0d0a59c438d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:11.027850 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ec1c6780-5bb1-4808-b89a-c0d0a59c438d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:11.035809 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ec1c6780-5bb1-4808-b89a-c0d0a59c438d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:11.036818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ec1c6780-5bb1-4808-b89a-c0d0a59c438d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:11.037602 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ec1c6780-5bb1-4808-b89a-c0d0a59c438d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:11.038775 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ec1c6780-5bb1-4808-b89a-c0d0a59c438d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:11.039566 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ec1c6780-5bb1-4808-b89a-c0d0a59c438d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:11.040314 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ec1c6780-5bb1-4808-b89a-c0d0a59c438d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:11.090185 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-ec1c6780-5bb1-4808-b89a-c0d0a59c438d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lazy-loading 'flavor' on Instance uuid b68e375d-f7ab-4d35-a8c7-081596d0e6e2 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:51:11.413830 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ec1c6780-5bb1-4808-b89a-c0d0a59c438d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "POST /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/action" status: 202 len: 0 microversion: 2.20 time: 0.394880 Oct 28 09:51:11.415262 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 419/831] 10.222.0.25 () {62 vars in 1319 bytes} [Fri Oct 28 09:51:11 2022] POST /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/action => generated 0 bytes in 397 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:51:11.425553 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-95507e0a-051d-4d75-8d18-678c9f2cacc2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:11.427454 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-95507e0a-051d-4d75-8d18-678c9f2cacc2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:11.537419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-95507e0a-051d-4d75-8d18-678c9f2cacc2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:11.549692 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-95507e0a-051d-4d75-8d18-678c9f2cacc2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1241 microversion: 2.1 time: 0.131993 Oct 28 09:51:11.550755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 420/832] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:11 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1241 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:51:11.572801 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a426d382-9ba4-4480-beaa-af1a2a09719b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:11.573777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a426d382-9ba4-4480-beaa-af1a2a09719b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:11.585836 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a426d382-9ba4-4480-beaa-af1a2a09719b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:11.587207 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a426d382-9ba4-4480-beaa-af1a2a09719b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:11.587865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a426d382-9ba4-4480-beaa-af1a2a09719b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:11.588925 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a426d382-9ba4-4480-beaa-af1a2a09719b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:11.590219 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a426d382-9ba4-4480-beaa-af1a2a09719b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:11.591074 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a426d382-9ba4-4480-beaa-af1a2a09719b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:11.819072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a426d382-9ba4-4480-beaa-af1a2a09719b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:11.833230 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a426d382-9ba4-4480-beaa-af1a2a09719b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:11.836893 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a426d382-9ba4-4480-beaa-af1a2a09719b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:11.839529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a426d382-9ba4-4480-beaa-af1a2a09719b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.286502 Oct 28 09:51:11.841328 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 421/833] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:11 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:51:12.581178 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f818f01d-0caf-44a4-bdd4-4ab612ce4bcd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:12.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f818f01d-0caf-44a4-bdd4-4ab612ce4bcd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:12.592523 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f818f01d-0caf-44a4-bdd4-4ab612ce4bcd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:12.593869 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f818f01d-0caf-44a4-bdd4-4ab612ce4bcd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:12.594654 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f818f01d-0caf-44a4-bdd4-4ab612ce4bcd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:12.595866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f818f01d-0caf-44a4-bdd4-4ab612ce4bcd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:12.596523 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f818f01d-0caf-44a4-bdd4-4ab612ce4bcd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:12.597125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f818f01d-0caf-44a4-bdd4-4ab612ce4bcd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:12.737591 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f818f01d-0caf-44a4-bdd4-4ab612ce4bcd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:12.756730 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f818f01d-0caf-44a4-bdd4-4ab612ce4bcd tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1293 microversion: 2.1 time: 0.182689 Oct 28 09:51:12.775134 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 422/834] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:12 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1293 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:51:12.895446 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-68fe3ff1-0f76-4ec1-b799-cf5eada91a70 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:12.896682 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-68fe3ff1-0f76-4ec1-b799-cf5eada91a70 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:12.912696 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-68fe3ff1-0f76-4ec1-b799-cf5eada91a70 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:12.914155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-68fe3ff1-0f76-4ec1-b799-cf5eada91a70 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:12.915321 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-68fe3ff1-0f76-4ec1-b799-cf5eada91a70 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:12.916477 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-68fe3ff1-0f76-4ec1-b799-cf5eada91a70 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:12.917472 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-68fe3ff1-0f76-4ec1-b799-cf5eada91a70 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:12.918180 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-68fe3ff1-0f76-4ec1-b799-cf5eada91a70 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:13.199552 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-68fe3ff1-0f76-4ec1-b799-cf5eada91a70 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:13.210610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-68fe3ff1-0f76-4ec1-b799-cf5eada91a70 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:13.213418 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-68fe3ff1-0f76-4ec1-b799-cf5eada91a70 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:13.214648 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-68fe3ff1-0f76-4ec1-b799-cf5eada91a70 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.333233 Oct 28 09:51:13.215621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 423/835] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:12 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:51:13.795440 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c34ce481-02a5-487a-a39b-3dfffbf05009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:13.797996 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c34ce481-02a5-487a-a39b-3dfffbf05009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:13.800233 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:13.811169 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c34ce481-02a5-487a-a39b-3dfffbf05009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:13.812111 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c34ce481-02a5-487a-a39b-3dfffbf05009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:13.813503 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c34ce481-02a5-487a-a39b-3dfffbf05009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:13.814832 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c34ce481-02a5-487a-a39b-3dfffbf05009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:13.815581 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c34ce481-02a5-487a-a39b-3dfffbf05009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:13.816338 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c34ce481-02a5-487a-a39b-3dfffbf05009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:13.950202 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c34ce481-02a5-487a-a39b-3dfffbf05009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:13.964816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c34ce481-02a5-487a-a39b-3dfffbf05009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1299 microversion: 2.1 time: 0.174489 Oct 28 09:51:13.965895 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 424/836] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:13 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1299 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:51:14.241330 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-39f0347e-417c-42eb-8156-e0dbd70053bf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:14.242353 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-39f0347e-417c-42eb-8156-e0dbd70053bf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:14.249798 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39f0347e-417c-42eb-8156-e0dbd70053bf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:14.249798 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39f0347e-417c-42eb-8156-e0dbd70053bf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:14.250290 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39f0347e-417c-42eb-8156-e0dbd70053bf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:14.250826 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39f0347e-417c-42eb-8156-e0dbd70053bf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:14.251279 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39f0347e-417c-42eb-8156-e0dbd70053bf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:14.251727 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39f0347e-417c-42eb-8156-e0dbd70053bf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:14.580100 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-39f0347e-417c-42eb-8156-e0dbd70053bf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:14.595689 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-39f0347e-417c-42eb-8156-e0dbd70053bf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:14.599348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-39f0347e-417c-42eb-8156-e0dbd70053bf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:14.602032 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-39f0347e-417c-42eb-8156-e0dbd70053bf tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.364759 Oct 28 09:51:14.603125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 425/837] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:14 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:15.009761 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c2e1aade-b0aa-4a6f-8e4d-e3f7548e6dc4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:15.012380 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c2e1aade-b0aa-4a6f-8e4d-e3f7548e6dc4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:15.025965 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c2e1aade-b0aa-4a6f-8e4d-e3f7548e6dc4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:15.027639 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c2e1aade-b0aa-4a6f-8e4d-e3f7548e6dc4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:15.028776 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c2e1aade-b0aa-4a6f-8e4d-e3f7548e6dc4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:15.030053 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c2e1aade-b0aa-4a6f-8e4d-e3f7548e6dc4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:15.030974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c2e1aade-b0aa-4a6f-8e4d-e3f7548e6dc4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:15.031835 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c2e1aade-b0aa-4a6f-8e4d-e3f7548e6dc4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:15.308098 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c2e1aade-b0aa-4a6f-8e4d-e3f7548e6dc4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:15.324261 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c2e1aade-b0aa-4a6f-8e4d-e3f7548e6dc4 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1366 microversion: 2.1 time: 0.321642 Oct 28 09:51:15.325276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 426/838] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:15 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1366 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:51:15.450990 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-2f477e07-36d5-4076-b2a6-2ceb85f55106 req-5a6ff2c7-b53e-4d56-87f0-7a4420f55a1a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9d28bd27-0d1c-45d2-8d9d-81595af44f70", "tag": "ecc67eeb-ed37-4710-a977-acceff867b07"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:15.464706 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-2f477e07-36d5-4076-b2a6-2ceb85f55106 req-5a6ff2c7-b53e-4d56-87f0-7a4420f55a1a service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:15.465206 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-2f477e07-36d5-4076-b2a6-2ceb85f55106 req-5a6ff2c7-b53e-4d56-87f0-7a4420f55a1a service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:15.465971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-2f477e07-36d5-4076-b2a6-2ceb85f55106 req-5a6ff2c7-b53e-4d56-87f0-7a4420f55a1a service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:15.485814 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-2f477e07-36d5-4076-b2a6-2ceb85f55106 req-5a6ff2c7-b53e-4d56-87f0-7a4420f55a1a service nova] Creating event network-changed:ecc67eeb-ed37-4710-a977-acceff867b07 for instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 on hv-cinder-zed-3 Oct 28 09:51:15.493038 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-2f477e07-36d5-4076-b2a6-2ceb85f55106 req-5a6ff2c7-b53e-4d56-87f0-7a4420f55a1a service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050105 Oct 28 09:51:15.493907 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 427/839] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:51:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:51:15.634206 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-cbd4e105-2b84-4041-85f9-735805f9341f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:15.635479 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-cbd4e105-2b84-4041-85f9-735805f9341f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:15.648455 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbd4e105-2b84-4041-85f9-735805f9341f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:15.649183 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbd4e105-2b84-4041-85f9-735805f9341f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:15.649926 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbd4e105-2b84-4041-85f9-735805f9341f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:15.651228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbd4e105-2b84-4041-85f9-735805f9341f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:15.652085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbd4e105-2b84-4041-85f9-735805f9341f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:15.652914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbd4e105-2b84-4041-85f9-735805f9341f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:15.918906 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-cbd4e105-2b84-4041-85f9-735805f9341f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:15.930894 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-cbd4e105-2b84-4041-85f9-735805f9341f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:15.932812 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-cbd4e105-2b84-4041-85f9-735805f9341f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:15.934030 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-cbd4e105-2b84-4041-85f9-735805f9341f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.305319 Oct 28 09:51:15.934712 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 428/840] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:15 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:51:16.353663 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b6766c2d-3192-4c77-9675-be74530811e2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:16.354865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b6766c2d-3192-4c77-9675-be74530811e2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:16.366931 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6766c2d-3192-4c77-9675-be74530811e2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:16.369012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6766c2d-3192-4c77-9675-be74530811e2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:16.369784 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6766c2d-3192-4c77-9675-be74530811e2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:16.371086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6766c2d-3192-4c77-9675-be74530811e2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:16.371904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6766c2d-3192-4c77-9675-be74530811e2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:16.372644 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6766c2d-3192-4c77-9675-be74530811e2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:16.530078 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b6766c2d-3192-4c77-9675-be74530811e2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:16.539248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b6766c2d-3192-4c77-9675-be74530811e2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1366 microversion: 2.1 time: 0.192825 Oct 28 09:51:16.539842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 429/841] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:16 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1366 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:51:16.965893 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7f5b4157-9d22-4b87-8964-5ec245fd3834 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:16.967354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7f5b4157-9d22-4b87-8964-5ec245fd3834 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:16.977328 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7f5b4157-9d22-4b87-8964-5ec245fd3834 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:16.978936 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7f5b4157-9d22-4b87-8964-5ec245fd3834 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:16.979987 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7f5b4157-9d22-4b87-8964-5ec245fd3834 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:16.981207 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7f5b4157-9d22-4b87-8964-5ec245fd3834 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:16.981988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7f5b4157-9d22-4b87-8964-5ec245fd3834 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:16.983031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7f5b4157-9d22-4b87-8964-5ec245fd3834 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:17.351114 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7f5b4157-9d22-4b87-8964-5ec245fd3834 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:17.360504 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7f5b4157-9d22-4b87-8964-5ec245fd3834 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:17.362006 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7f5b4157-9d22-4b87-8964-5ec245fd3834 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:17.363276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7f5b4157-9d22-4b87-8964-5ec245fd3834 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.405627 Oct 28 09:51:17.363867 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 430/842] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:16 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:51:17.566979 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-64dd26d3-a178-4e06-adc5-b2dd905b861e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:17.568454 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-64dd26d3-a178-4e06-adc5-b2dd905b861e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:17.586090 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64dd26d3-a178-4e06-adc5-b2dd905b861e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:17.586750 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64dd26d3-a178-4e06-adc5-b2dd905b861e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:17.587249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64dd26d3-a178-4e06-adc5-b2dd905b861e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:17.588061 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64dd26d3-a178-4e06-adc5-b2dd905b861e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:17.588597 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64dd26d3-a178-4e06-adc5-b2dd905b861e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:17.589249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-64dd26d3-a178-4e06-adc5-b2dd905b861e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:17.840249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-64dd26d3-a178-4e06-adc5-b2dd905b861e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:17.856442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-64dd26d3-a178-4e06-adc5-b2dd905b861e tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1366 microversion: 2.1 time: 0.293982 Oct 28 09:51:17.857512 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 431/843] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:17 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1366 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:51:18.393683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-091b0b0a-23d4-49e9-9237-79ce2924e4d5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:18.394813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-091b0b0a-23d4-49e9-9237-79ce2924e4d5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:18.408861 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-091b0b0a-23d4-49e9-9237-79ce2924e4d5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:18.409530 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-091b0b0a-23d4-49e9-9237-79ce2924e4d5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:18.409857 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-091b0b0a-23d4-49e9-9237-79ce2924e4d5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:18.410546 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-091b0b0a-23d4-49e9-9237-79ce2924e4d5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:18.411006 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-091b0b0a-23d4-49e9-9237-79ce2924e4d5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:18.411436 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-091b0b0a-23d4-49e9-9237-79ce2924e4d5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:18.626980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-091b0b0a-23d4-49e9-9237-79ce2924e4d5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:18.663975 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-091b0b0a-23d4-49e9-9237-79ce2924e4d5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:18.666805 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-091b0b0a-23d4-49e9-9237-79ce2924e4d5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:18.669121 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-091b0b0a-23d4-49e9-9237-79ce2924e4d5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.282906 Oct 28 09:51:18.670137 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 432/844] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:18 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:51:18.901461 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-42640f27-c929-423a-936f-e0663b8ec463 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:18.902481 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-42640f27-c929-423a-936f-e0663b8ec463 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:18.914074 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-42640f27-c929-423a-936f-e0663b8ec463 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:18.915014 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-42640f27-c929-423a-936f-e0663b8ec463 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:18.915813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-42640f27-c929-423a-936f-e0663b8ec463 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:18.917139 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-42640f27-c929-423a-936f-e0663b8ec463 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:18.917980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-42640f27-c929-423a-936f-e0663b8ec463 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:18.918788 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-42640f27-c929-423a-936f-e0663b8ec463 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:18.949596 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:19.115789 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-42640f27-c929-423a-936f-e0663b8ec463 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:19.124335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-42640f27-c929-423a-936f-e0663b8ec463 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1366 microversion: 2.1 time: 0.230401 Oct 28 09:51:19.125042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 433/845] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:18 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1366 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:51:19.700549 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3c6516a1-91ed-4776-a197-1e499460470b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:19.701720 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3c6516a1-91ed-4776-a197-1e499460470b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:19.714610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c6516a1-91ed-4776-a197-1e499460470b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:19.715522 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c6516a1-91ed-4776-a197-1e499460470b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:19.716307 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c6516a1-91ed-4776-a197-1e499460470b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:19.717602 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c6516a1-91ed-4776-a197-1e499460470b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:19.719031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c6516a1-91ed-4776-a197-1e499460470b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:19.719797 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c6516a1-91ed-4776-a197-1e499460470b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:20.099147 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3c6516a1-91ed-4776-a197-1e499460470b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:20.118642 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3c6516a1-91ed-4776-a197-1e499460470b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:20.121919 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3c6516a1-91ed-4776-a197-1e499460470b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:20.124334 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3c6516a1-91ed-4776-a197-1e499460470b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.431323 Oct 28 09:51:20.125997 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 434/846] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:19 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:20.149554 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-fe3f4466-c7a9-4aa8-bb1c-b4d3211be9c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:20.150725 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-fe3f4466-c7a9-4aa8-bb1c-b4d3211be9c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:20.158883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fe3f4466-c7a9-4aa8-bb1c-b4d3211be9c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:20.159609 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fe3f4466-c7a9-4aa8-bb1c-b4d3211be9c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:20.160058 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fe3f4466-c7a9-4aa8-bb1c-b4d3211be9c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:20.160798 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fe3f4466-c7a9-4aa8-bb1c-b4d3211be9c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:20.161254 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fe3f4466-c7a9-4aa8-bb1c-b4d3211be9c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:20.161939 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fe3f4466-c7a9-4aa8-bb1c-b4d3211be9c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:20.385859 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-fe3f4466-c7a9-4aa8-bb1c-b4d3211be9c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:20.403785 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-fe3f4466-c7a9-4aa8-bb1c-b4d3211be9c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1366 microversion: 2.1 time: 0.258629 Oct 28 09:51:20.421208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 435/847] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:20 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1366 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:51:20.486171 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-2865dd02-4632-45a7-a14f-21cc2c6a1be8 req-8acc3f8a-786a-422d-b197-ff01e1a611cc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9d28bd27-0d1c-45d2-8d9d-81595af44f70", "name": "network-vif-plugged", "status": "completed", "tag": "ecc67eeb-ed37-4710-a977-acceff867b07"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:20.501336 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-2865dd02-4632-45a7-a14f-21cc2c6a1be8 req-8acc3f8a-786a-422d-b197-ff01e1a611cc service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:20.502425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-2865dd02-4632-45a7-a14f-21cc2c6a1be8 req-8acc3f8a-786a-422d-b197-ff01e1a611cc service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:20.503101 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-2865dd02-4632-45a7-a14f-21cc2c6a1be8 req-8acc3f8a-786a-422d-b197-ff01e1a611cc service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:20.520782 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-2865dd02-4632-45a7-a14f-21cc2c6a1be8 req-8acc3f8a-786a-422d-b197-ff01e1a611cc service nova] Creating event network-vif-plugged:ecc67eeb-ed37-4710-a977-acceff867b07 for instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 on hv-cinder-zed-3 Oct 28 09:51:20.527953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-2865dd02-4632-45a7-a14f-21cc2c6a1be8 req-8acc3f8a-786a-422d-b197-ff01e1a611cc service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046369 Oct 28 09:51:20.529264 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 436/848] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:51:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:51:21.168775 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:21.175971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9a1e1642-121b-43d6-b5f5-7b1f48a66b3e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:21.177210 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9a1e1642-121b-43d6-b5f5-7b1f48a66b3e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:21.188685 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9a1e1642-121b-43d6-b5f5-7b1f48a66b3e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:21.189675 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9a1e1642-121b-43d6-b5f5-7b1f48a66b3e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:21.190375 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9a1e1642-121b-43d6-b5f5-7b1f48a66b3e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:21.191489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9a1e1642-121b-43d6-b5f5-7b1f48a66b3e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:21.192042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9a1e1642-121b-43d6-b5f5-7b1f48a66b3e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:21.192954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9a1e1642-121b-43d6-b5f5-7b1f48a66b3e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:21.410871 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-9a1e1642-121b-43d6-b5f5-7b1f48a66b3e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:21.420072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-9a1e1642-121b-43d6-b5f5-7b1f48a66b3e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:21.422299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-9a1e1642-121b-43d6-b5f5-7b1f48a66b3e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:21.423921 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9a1e1642-121b-43d6-b5f5-7b1f48a66b3e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.256678 Oct 28 09:51:21.424489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 437/849] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:21 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:21.438030 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-95af5226-4e09-46bd-acd7-6cd5119c3c27 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:21.438923 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-95af5226-4e09-46bd-acd7-6cd5119c3c27 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:21.446020 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95af5226-4e09-46bd-acd7-6cd5119c3c27 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:21.446545 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95af5226-4e09-46bd-acd7-6cd5119c3c27 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:21.446985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95af5226-4e09-46bd-acd7-6cd5119c3c27 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:21.447706 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95af5226-4e09-46bd-acd7-6cd5119c3c27 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:21.448222 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95af5226-4e09-46bd-acd7-6cd5119c3c27 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:21.448649 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95af5226-4e09-46bd-acd7-6cd5119c3c27 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:21.549904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=12 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:21.607496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-95af5226-4e09-46bd-acd7-6cd5119c3c27 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:21.617153 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-95af5226-4e09-46bd-acd7-6cd5119c3c27 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1366 microversion: 2.1 time: 0.183544 Oct 28 09:51:21.617749 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 438/850] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:21 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1366 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:51:22.468913 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d9ef1bc8-ee92-4135-9ad2-3cf82702f5cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:22.469909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d9ef1bc8-ee92-4135-9ad2-3cf82702f5cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:22.476467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d9ef1bc8-ee92-4135-9ad2-3cf82702f5cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:22.476952 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d9ef1bc8-ee92-4135-9ad2-3cf82702f5cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:22.477705 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d9ef1bc8-ee92-4135-9ad2-3cf82702f5cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:22.478466 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d9ef1bc8-ee92-4135-9ad2-3cf82702f5cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:22.478923 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d9ef1bc8-ee92-4135-9ad2-3cf82702f5cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:22.479345 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d9ef1bc8-ee92-4135-9ad2-3cf82702f5cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:22.733413 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d9ef1bc8-ee92-4135-9ad2-3cf82702f5cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:22.748236 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d9ef1bc8-ee92-4135-9ad2-3cf82702f5cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:22.751381 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d9ef1bc8-ee92-4135-9ad2-3cf82702f5cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:22.753370 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d9ef1bc8-ee92-4135-9ad2-3cf82702f5cc tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.289161 Oct 28 09:51:22.754305 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 439/851] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:22 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:22.763080 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-99e7f0ba-57ce-4485-ad89-f467cf3e02d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:22.764230 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-99e7f0ba-57ce-4485-ad89-f467cf3e02d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:22.775341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-99e7f0ba-57ce-4485-ad89-f467cf3e02d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:22.776220 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-99e7f0ba-57ce-4485-ad89-f467cf3e02d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:22.778313 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-99e7f0ba-57ce-4485-ad89-f467cf3e02d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:22.779933 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-99e7f0ba-57ce-4485-ad89-f467cf3e02d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:22.781052 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-99e7f0ba-57ce-4485-ad89-f467cf3e02d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:22.782065 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-99e7f0ba-57ce-4485-ad89-f467cf3e02d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:22.958524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-99e7f0ba-57ce-4485-ad89-f467cf3e02d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:22.972742 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-99e7f0ba-57ce-4485-ad89-f467cf3e02d9 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1557 microversion: 2.1 time: 0.216454 Oct 28 09:51:22.973833 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 440/852] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:51:22 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1557 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:51:22.997399 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e29e16c3-da65-4328-9659-8aa3ec70d9de tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:22.998157 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e29e16c3-da65-4328-9659-8aa3ec70d9de tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:23.004563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e29e16c3-da65-4328-9659-8aa3ec70d9de tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:23.005258 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e29e16c3-da65-4328-9659-8aa3ec70d9de tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:23.005758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e29e16c3-da65-4328-9659-8aa3ec70d9de tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:23.006515 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e29e16c3-da65-4328-9659-8aa3ec70d9de tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:23.006969 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e29e16c3-da65-4328-9659-8aa3ec70d9de tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:23.007386 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e29e16c3-da65-4328-9659-8aa3ec70d9de tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:23.107534 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e29e16c3-da65-4328-9659-8aa3ec70d9de tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.114220 Oct 28 09:51:23.108418 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 441/853] 10.222.0.25 () {58 vars in 1275 bytes} [Fri Oct 28 09:51:22 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/os-interface => generated 285 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:51:23.803094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-59301134-1ef3-4d97-8beb-69334baea414 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:23.804108 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-59301134-1ef3-4d97-8beb-69334baea414 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:23.814683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59301134-1ef3-4d97-8beb-69334baea414 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:23.815396 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59301134-1ef3-4d97-8beb-69334baea414 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:23.815976 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59301134-1ef3-4d97-8beb-69334baea414 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:23.817059 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59301134-1ef3-4d97-8beb-69334baea414 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:23.817766 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59301134-1ef3-4d97-8beb-69334baea414 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:23.818487 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59301134-1ef3-4d97-8beb-69334baea414 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:24.064404 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-59301134-1ef3-4d97-8beb-69334baea414 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:24.079763 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-59301134-1ef3-4d97-8beb-69334baea414 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:24.082297 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-59301134-1ef3-4d97-8beb-69334baea414 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:24.086486 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-59301134-1ef3-4d97-8beb-69334baea414 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.291476 Oct 28 09:51:24.090504 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 442/854] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:23 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:24.484200 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-bc4d6c57-a093-4749-b760-7b213b807065 req-df962a38-baab-404d-afed-32c7bf38ef43 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9d28bd27-0d1c-45d2-8d9d-81595af44f70", "tag": "ecc67eeb-ed37-4710-a977-acceff867b07"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:24.495367 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-bc4d6c57-a093-4749-b760-7b213b807065 req-df962a38-baab-404d-afed-32c7bf38ef43 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:24.496428 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-bc4d6c57-a093-4749-b760-7b213b807065 req-df962a38-baab-404d-afed-32c7bf38ef43 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:24.497043 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-bc4d6c57-a093-4749-b760-7b213b807065 req-df962a38-baab-404d-afed-32c7bf38ef43 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:24.511962 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-bc4d6c57-a093-4749-b760-7b213b807065 req-df962a38-baab-404d-afed-32c7bf38ef43 service nova] Creating event network-changed:ecc67eeb-ed37-4710-a977-acceff867b07 for instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 on hv-cinder-zed-3 Oct 28 09:51:24.519128 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-bc4d6c57-a093-4749-b760-7b213b807065 req-df962a38-baab-404d-afed-32c7bf38ef43 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038947 Oct 28 09:51:24.520084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 443/855] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:51:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:51:25.128277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e7305b3a-1dbb-4e95-a3c4-ff485614a396 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:25.129551 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e7305b3a-1dbb-4e95-a3c4-ff485614a396 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:25.144570 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e7305b3a-1dbb-4e95-a3c4-ff485614a396 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:25.145975 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e7305b3a-1dbb-4e95-a3c4-ff485614a396 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:25.147179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e7305b3a-1dbb-4e95-a3c4-ff485614a396 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:25.148870 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e7305b3a-1dbb-4e95-a3c4-ff485614a396 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:25.149763 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e7305b3a-1dbb-4e95-a3c4-ff485614a396 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:25.151881 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e7305b3a-1dbb-4e95-a3c4-ff485614a396 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:25.533820 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e7305b3a-1dbb-4e95-a3c4-ff485614a396 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:25.543587 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e7305b3a-1dbb-4e95-a3c4-ff485614a396 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:25.545743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e7305b3a-1dbb-4e95-a3c4-ff485614a396 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:25.547237 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e7305b3a-1dbb-4e95-a3c4-ff485614a396 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.429047 Oct 28 09:51:25.548363 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 444/856] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:25 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:51:26.593190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f11bb67f-b89e-4b9e-9f18-d35b203a4c72 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:26.594830 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f11bb67f-b89e-4b9e-9f18-d35b203a4c72 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:26.601807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f11bb67f-b89e-4b9e-9f18-d35b203a4c72 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:26.602293 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f11bb67f-b89e-4b9e-9f18-d35b203a4c72 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:26.602757 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f11bb67f-b89e-4b9e-9f18-d35b203a4c72 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:26.603477 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f11bb67f-b89e-4b9e-9f18-d35b203a4c72 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:26.603929 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f11bb67f-b89e-4b9e-9f18-d35b203a4c72 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:26.604348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f11bb67f-b89e-4b9e-9f18-d35b203a4c72 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:26.766654 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f11bb67f-b89e-4b9e-9f18-d35b203a4c72 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:26.776520 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f11bb67f-b89e-4b9e-9f18-d35b203a4c72 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:26.778228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f11bb67f-b89e-4b9e-9f18-d35b203a4c72 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:26.779906 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f11bb67f-b89e-4b9e-9f18-d35b203a4c72 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.191452 Oct 28 09:51:26.781108 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 445/857] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:26 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:27.817861 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e9cd231b-31e4-4d8e-ba8c-d49fca3e758a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:27.819400 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e9cd231b-31e4-4d8e-ba8c-d49fca3e758a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:27.829232 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e9cd231b-31e4-4d8e-ba8c-d49fca3e758a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:27.829884 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e9cd231b-31e4-4d8e-ba8c-d49fca3e758a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:27.830317 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e9cd231b-31e4-4d8e-ba8c-d49fca3e758a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:27.831097 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e9cd231b-31e4-4d8e-ba8c-d49fca3e758a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:27.831705 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e9cd231b-31e4-4d8e-ba8c-d49fca3e758a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:27.832046 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e9cd231b-31e4-4d8e-ba8c-d49fca3e758a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:28.036160 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e9cd231b-31e4-4d8e-ba8c-d49fca3e758a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:28.052316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e9cd231b-31e4-4d8e-ba8c-d49fca3e758a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:28.058282 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e9cd231b-31e4-4d8e-ba8c-d49fca3e758a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:28.060793 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e9cd231b-31e4-4d8e-ba8c-d49fca3e758a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1743 microversion: 2.20 time: 0.247044 Oct 28 09:51:28.061953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 446/858] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:27 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1743 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:29.095104 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1648fa21-85f1-4a31-b5b2-b533f7760b3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:29.095810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1648fa21-85f1-4a31-b5b2-b533f7760b3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:29.103100 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1648fa21-85f1-4a31-b5b2-b533f7760b3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:29.103908 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1648fa21-85f1-4a31-b5b2-b533f7760b3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:29.104478 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1648fa21-85f1-4a31-b5b2-b533f7760b3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:29.105364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1648fa21-85f1-4a31-b5b2-b533f7760b3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:29.106038 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1648fa21-85f1-4a31-b5b2-b533f7760b3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:29.106724 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1648fa21-85f1-4a31-b5b2-b533f7760b3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:29.261257 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1648fa21-85f1-4a31-b5b2-b533f7760b3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:29.272172 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1648fa21-85f1-4a31-b5b2-b533f7760b3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:29.274884 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1648fa21-85f1-4a31-b5b2-b533f7760b3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:29.277341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1648fa21-85f1-4a31-b5b2-b533f7760b3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1743 microversion: 2.20 time: 0.186477 Oct 28 09:51:29.278131 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 447/859] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:29 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1743 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:30.311708 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7a9d565f-628d-4339-9672-bad368765f1a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:30.314567 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7a9d565f-628d-4339-9672-bad368765f1a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:30.322896 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a9d565f-628d-4339-9672-bad368765f1a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:30.323768 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a9d565f-628d-4339-9672-bad368765f1a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:30.324630 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a9d565f-628d-4339-9672-bad368765f1a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:30.325609 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a9d565f-628d-4339-9672-bad368765f1a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:30.326292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a9d565f-628d-4339-9672-bad368765f1a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:30.326979 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a9d565f-628d-4339-9672-bad368765f1a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:30.619267 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7a9d565f-628d-4339-9672-bad368765f1a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:30.643156 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7a9d565f-628d-4339-9672-bad368765f1a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:30.646961 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7a9d565f-628d-4339-9672-bad368765f1a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:30.649046 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7a9d565f-628d-4339-9672-bad368765f1a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1738 microversion: 2.20 time: 0.347713 Oct 28 09:51:30.649997 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 448/860] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:30 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1738 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:51:30.659469 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-39e6f342-efef-4a83-b06e-b15fbf82941e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:30.660930 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-39e6f342-efef-4a83-b06e-b15fbf82941e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:30.684484 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39e6f342-efef-4a83-b06e-b15fbf82941e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:30.685301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39e6f342-efef-4a83-b06e-b15fbf82941e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:30.686039 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39e6f342-efef-4a83-b06e-b15fbf82941e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:30.687239 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39e6f342-efef-4a83-b06e-b15fbf82941e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:30.688214 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39e6f342-efef-4a83-b06e-b15fbf82941e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:30.688881 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39e6f342-efef-4a83-b06e-b15fbf82941e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:30.742348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-39e6f342-efef-4a83-b06e-b15fbf82941e tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/ips" status: 200 len: 146 microversion: 2.1 time: 0.090257 Oct 28 09:51:30.744466 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 449/861] 10.222.0.25 () {58 vars in 1248 bytes} [Fri Oct 28 09:51:30 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/ips => generated 146 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:51:30.754952 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-9bd1f810-69a6-45f5-b600-f3e6a73a5f77 req-25f63c1d-24d7-4f06-9c82-7da38c183cd6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b68e375d-f7ab-4d35-a8c7-081596d0e6e2", "name": "network-vif-plugged", "status": "completed", "tag": "533de8d8-0438-4dd2-946d-f12312377670"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:30.774377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-9bd1f810-69a6-45f5-b600-f3e6a73a5f77 req-25f63c1d-24d7-4f06-9c82-7da38c183cd6 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:30.775563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-9bd1f810-69a6-45f5-b600-f3e6a73a5f77 req-25f63c1d-24d7-4f06-9c82-7da38c183cd6 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:30.776450 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-9bd1f810-69a6-45f5-b600-f3e6a73a5f77 req-25f63c1d-24d7-4f06-9c82-7da38c183cd6 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:30.804571 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-9bd1f810-69a6-45f5-b600-f3e6a73a5f77 req-25f63c1d-24d7-4f06-9c82-7da38c183cd6 service nova] Creating event network-vif-plugged:533de8d8-0438-4dd2-946d-f12312377670 for instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 on hv-cinder-zed-3 Oct 28 09:51:30.811154 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-9bd1f810-69a6-45f5-b600-f3e6a73a5f77 req-25f63c1d-24d7-4f06-9c82-7da38c183cd6 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063522 Oct 28 09:51:30.813096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 450/862] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:51:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:51:31.703264 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f8d5f0fe-e665-41dc-b982-fa5e66a53d0a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:31.704523 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f8d5f0fe-e665-41dc-b982-fa5e66a53d0a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:31.716198 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f8d5f0fe-e665-41dc-b982-fa5e66a53d0a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:31.717591 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f8d5f0fe-e665-41dc-b982-fa5e66a53d0a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:31.718906 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f8d5f0fe-e665-41dc-b982-fa5e66a53d0a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:31.720478 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f8d5f0fe-e665-41dc-b982-fa5e66a53d0a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:31.721683 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f8d5f0fe-e665-41dc-b982-fa5e66a53d0a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:31.722847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f8d5f0fe-e665-41dc-b982-fa5e66a53d0a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:32.008646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f8d5f0fe-e665-41dc-b982-fa5e66a53d0a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:32.024073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f8d5f0fe-e665-41dc-b982-fa5e66a53d0a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:32.027134 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f8d5f0fe-e665-41dc-b982-fa5e66a53d0a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:32.029397 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f8d5f0fe-e665-41dc-b982-fa5e66a53d0a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1738 microversion: 2.20 time: 0.334439 Oct 28 09:51:32.031144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 451/863] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:31 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1738 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:32.542315 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-28b00852-40ef-4122-8d66-53803ceb9029 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ba5635e0-5253-46eb-a1c7-a099b5af6518"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:32.546983 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-28b00852-40ef-4122-8d66-53803ceb9029 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:32.555773 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-28b00852-40ef-4122-8d66-53803ceb9029 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:32.556974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-28b00852-40ef-4122-8d66-53803ceb9029 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:32.558378 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-28b00852-40ef-4122-8d66-53803ceb9029 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:32.561263 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-28b00852-40ef-4122-8d66-53803ceb9029 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:32.562228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-28b00852-40ef-4122-8d66-53803ceb9029 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:32.562702 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-28b00852-40ef-4122-8d66-53803ceb9029 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:32.681136 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=100 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:33.049996 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-28b00852-40ef-4122-8d66-53803ceb9029 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "POST /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.513361 Oct 28 09:51:33.054178 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 452/864] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:51:32 2022] POST /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments => generated 194 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:51:33.090445 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a8b83cd4-1404-42bb-ba96-c05c5df37ca5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:33.090445 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a8b83cd4-1404-42bb-ba96-c05c5df37ca5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:33.103813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a8b83cd4-1404-42bb-ba96-c05c5df37ca5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:33.104969 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a8b83cd4-1404-42bb-ba96-c05c5df37ca5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:33.105901 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a8b83cd4-1404-42bb-ba96-c05c5df37ca5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:33.107637 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a8b83cd4-1404-42bb-ba96-c05c5df37ca5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:33.109248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a8b83cd4-1404-42bb-ba96-c05c5df37ca5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:33.110465 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a8b83cd4-1404-42bb-ba96-c05c5df37ca5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:33.395400 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a8b83cd4-1404-42bb-ba96-c05c5df37ca5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:33.417301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a8b83cd4-1404-42bb-ba96-c05c5df37ca5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:33.421728 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a8b83cd4-1404-42bb-ba96-c05c5df37ca5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:33.423973 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a8b83cd4-1404-42bb-ba96-c05c5df37ca5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1738 microversion: 2.20 time: 0.349012 Oct 28 09:51:33.431755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 453/865] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:33 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1738 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:51:34.480169 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a0246f8d-1666-4f0f-8abd-cdf6f8600416 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:34.483033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a0246f8d-1666-4f0f-8abd-cdf6f8600416 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:34.497664 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a0246f8d-1666-4f0f-8abd-cdf6f8600416 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:34.498774 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a0246f8d-1666-4f0f-8abd-cdf6f8600416 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:34.501877 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a0246f8d-1666-4f0f-8abd-cdf6f8600416 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:34.503243 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a0246f8d-1666-4f0f-8abd-cdf6f8600416 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:34.504290 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a0246f8d-1666-4f0f-8abd-cdf6f8600416 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:34.505336 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a0246f8d-1666-4f0f-8abd-cdf6f8600416 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:34.775794 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a0246f8d-1666-4f0f-8abd-cdf6f8600416 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:34.789344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a0246f8d-1666-4f0f-8abd-cdf6f8600416 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:34.792725 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a0246f8d-1666-4f0f-8abd-cdf6f8600416 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:34.795327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a0246f8d-1666-4f0f-8abd-cdf6f8600416 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1738 microversion: 2.20 time: 0.325440 Oct 28 09:51:34.796690 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 454/866] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:34 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1738 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:51:35.852752 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6f529f1d-d635-4b66-b210-314bd24f8b45 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:35.853727 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6f529f1d-d635-4b66-b210-314bd24f8b45 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:35.861252 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f529f1d-d635-4b66-b210-314bd24f8b45 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:35.862153 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f529f1d-d635-4b66-b210-314bd24f8b45 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:35.862881 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f529f1d-d635-4b66-b210-314bd24f8b45 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:35.863823 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f529f1d-d635-4b66-b210-314bd24f8b45 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:35.864475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f529f1d-d635-4b66-b210-314bd24f8b45 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:35.865160 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f529f1d-d635-4b66-b210-314bd24f8b45 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:36.126094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6f529f1d-d635-4b66-b210-314bd24f8b45 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:36.144363 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6f529f1d-d635-4b66-b210-314bd24f8b45 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:36.147870 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6f529f1d-d635-4b66-b210-314bd24f8b45 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:36.150101 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6f529f1d-d635-4b66-b210-314bd24f8b45 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1738 microversion: 2.20 time: 0.302346 Oct 28 09:51:36.153817 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 455/867] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:35 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1738 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:37.204403 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-fc0f1148-f205-42bc-b323-a8e90a610810 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:37.205526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-fc0f1148-f205-42bc-b323-a8e90a610810 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:37.213827 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fc0f1148-f205-42bc-b323-a8e90a610810 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:37.214707 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fc0f1148-f205-42bc-b323-a8e90a610810 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:37.215525 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fc0f1148-f205-42bc-b323-a8e90a610810 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:37.216312 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fc0f1148-f205-42bc-b323-a8e90a610810 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:37.217039 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fc0f1148-f205-42bc-b323-a8e90a610810 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:37.217685 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fc0f1148-f205-42bc-b323-a8e90a610810 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:37.459732 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-fc0f1148-f205-42bc-b323-a8e90a610810 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:37.487525 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-fc0f1148-f205-42bc-b323-a8e90a610810 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:37.491248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-fc0f1148-f205-42bc-b323-a8e90a610810 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:37.493658 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-fc0f1148-f205-42bc-b323-a8e90a610810 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1738 microversion: 2.20 time: 0.293988 Oct 28 09:51:37.495526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 456/868] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:37 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1738 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:38.549233 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-84aed4e9-5ff6-462a-a4a6-a64e12ebaa97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:38.553955 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-84aed4e9-5ff6-462a-a4a6-a64e12ebaa97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:38.577823 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-84aed4e9-5ff6-462a-a4a6-a64e12ebaa97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:38.579234 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-84aed4e9-5ff6-462a-a4a6-a64e12ebaa97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:38.581435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-84aed4e9-5ff6-462a-a4a6-a64e12ebaa97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:38.583128 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-84aed4e9-5ff6-462a-a4a6-a64e12ebaa97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:38.590136 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-84aed4e9-5ff6-462a-a4a6-a64e12ebaa97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:38.592581 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-84aed4e9-5ff6-462a-a4a6-a64e12ebaa97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:38.804842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-84aed4e9-5ff6-462a-a4a6-a64e12ebaa97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:38.820294 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-84aed4e9-5ff6-462a-a4a6-a64e12ebaa97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:38.822840 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-84aed4e9-5ff6-462a-a4a6-a64e12ebaa97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:38.824384 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-84aed4e9-5ff6-462a-a4a6-a64e12ebaa97 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1738 microversion: 2.20 time: 0.291357 Oct 28 09:51:38.840389 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 457/869] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:38 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1738 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:51:39.852258 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-260c920d-b665-498d-ba78-16caf344986b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:39.853299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-260c920d-b665-498d-ba78-16caf344986b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:39.863091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-260c920d-b665-498d-ba78-16caf344986b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:39.863868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-260c920d-b665-498d-ba78-16caf344986b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:39.864858 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-260c920d-b665-498d-ba78-16caf344986b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:39.866042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-260c920d-b665-498d-ba78-16caf344986b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:39.866603 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-260c920d-b665-498d-ba78-16caf344986b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:39.867078 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-260c920d-b665-498d-ba78-16caf344986b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:40.107415 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-260c920d-b665-498d-ba78-16caf344986b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:40.119384 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-260c920d-b665-498d-ba78-16caf344986b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:40.120958 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-260c920d-b665-498d-ba78-16caf344986b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:40.122202 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-260c920d-b665-498d-ba78-16caf344986b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1738 microversion: 2.20 time: 0.275828 Oct 28 09:51:40.122817 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 458/870] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:39 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1738 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:41.130301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c9227fa1-4760-451b-87fe-b9f596a1aed7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:41.133707 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c9227fa1-4760-451b-87fe-b9f596a1aed7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:41.142034 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9227fa1-4760-451b-87fe-b9f596a1aed7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:41.147792 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9227fa1-4760-451b-87fe-b9f596a1aed7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:41.148521 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9227fa1-4760-451b-87fe-b9f596a1aed7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:41.149811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9227fa1-4760-451b-87fe-b9f596a1aed7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:41.150676 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9227fa1-4760-451b-87fe-b9f596a1aed7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:41.151290 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c9227fa1-4760-451b-87fe-b9f596a1aed7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:41.207432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c9227fa1-4760-451b-87fe-b9f596a1aed7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080755 Oct 28 09:51:41.208877 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 459/871] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:51:41 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments => generated 197 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:51:41.219001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7631bd42-9b2c-4cee-ae75-76a2313842c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:41.220478 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7631bd42-9b2c-4cee-ae75-76a2313842c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:41.231914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7631bd42-9b2c-4cee-ae75-76a2313842c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:41.233189 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7631bd42-9b2c-4cee-ae75-76a2313842c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:41.234833 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7631bd42-9b2c-4cee-ae75-76a2313842c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:41.236157 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7631bd42-9b2c-4cee-ae75-76a2313842c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:41.237225 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7631bd42-9b2c-4cee-ae75-76a2313842c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:41.238228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7631bd42-9b2c-4cee-ae75-76a2313842c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:41.492945 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7631bd42-9b2c-4cee-ae75-76a2313842c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:41.504880 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7631bd42-9b2c-4cee-ae75-76a2313842c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:41.506585 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7631bd42-9b2c-4cee-ae75-76a2313842c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:41.507923 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7631bd42-9b2c-4cee-ae75-76a2313842c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1738 microversion: 2.20 time: 0.297291 Oct 28 09:51:41.508601 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 460/872] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:41 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1738 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:51:41.522108 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-cd90785b-b723-4e70-b7cd-c14b72f44155 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:41.523040 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-cd90785b-b723-4e70-b7cd-c14b72f44155 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:41.533195 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd90785b-b723-4e70-b7cd-c14b72f44155 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:41.534109 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd90785b-b723-4e70-b7cd-c14b72f44155 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:41.534908 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd90785b-b723-4e70-b7cd-c14b72f44155 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:41.536224 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd90785b-b723-4e70-b7cd-c14b72f44155 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:41.537084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd90785b-b723-4e70-b7cd-c14b72f44155 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:41.537876 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd90785b-b723-4e70-b7cd-c14b72f44155 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:41.593695 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-cd90785b-b723-4e70-b7cd-c14b72f44155 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments/ba5635e0-5253-46eb-a1c7-a099b5af6518" status: 200 len: 194 microversion: 2.1 time: 0.078574 Oct 28 09:51:41.594572 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 461/873] 10.222.0.25 () {58 vars in 1413 bytes} [Fri Oct 28 09:51:41 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments/ba5635e0-5253-46eb-a1c7-a099b5af6518 => generated 194 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:51:42.551570 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b350a303-3216-4daf-b832-60f8cb588aef tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:42.552914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b350a303-3216-4daf-b832-60f8cb588aef tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:42.565770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b350a303-3216-4daf-b832-60f8cb588aef tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:42.567101 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b350a303-3216-4daf-b832-60f8cb588aef tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:42.568129 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b350a303-3216-4daf-b832-60f8cb588aef tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:42.572235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b350a303-3216-4daf-b832-60f8cb588aef tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:42.573343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b350a303-3216-4daf-b832-60f8cb588aef tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:42.574336 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b350a303-3216-4daf-b832-60f8cb588aef tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:42.777253 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b350a303-3216-4daf-b832-60f8cb588aef tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:42.792066 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b350a303-3216-4daf-b832-60f8cb588aef tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:42.796593 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b350a303-3216-4daf-b832-60f8cb588aef tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:42.799113 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b350a303-3216-4daf-b832-60f8cb588aef tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1738 microversion: 2.20 time: 0.258522 Oct 28 09:51:42.800329 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 462/874] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:42 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1738 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:51:43.182116 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0169ce57-33a4-4dad-b370-20036de304b7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "873eedf8-2b3f-4477-b37e-85d67d58c2ff"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:43.186014 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0169ce57-33a4-4dad-b370-20036de304b7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:43.196487 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0169ce57-33a4-4dad-b370-20036de304b7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:43.197054 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0169ce57-33a4-4dad-b370-20036de304b7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:43.197737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0169ce57-33a4-4dad-b370-20036de304b7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:43.198888 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0169ce57-33a4-4dad-b370-20036de304b7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:43.199601 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0169ce57-33a4-4dad-b370-20036de304b7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:43.200336 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0169ce57-33a4-4dad-b370-20036de304b7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:43.711886 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0169ce57-33a4-4dad-b370-20036de304b7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "POST /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.534699 Oct 28 09:51:43.712986 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 463/875] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:51:43 2022] POST /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments => generated 194 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:51:43.829474 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c211d80f-326a-41e8-a5a3-948594257f3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:43.830605 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c211d80f-326a-41e8-a5a3-948594257f3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:43.839627 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c211d80f-326a-41e8-a5a3-948594257f3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:43.840526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c211d80f-326a-41e8-a5a3-948594257f3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:43.841117 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c211d80f-326a-41e8-a5a3-948594257f3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:43.842241 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c211d80f-326a-41e8-a5a3-948594257f3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:43.843415 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c211d80f-326a-41e8-a5a3-948594257f3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:43.843976 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c211d80f-326a-41e8-a5a3-948594257f3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:44.129941 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c211d80f-326a-41e8-a5a3-948594257f3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:44.143387 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c211d80f-326a-41e8-a5a3-948594257f3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:44.146309 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c211d80f-326a-41e8-a5a3-948594257f3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:44.148207 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c211d80f-326a-41e8-a5a3-948594257f3b tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1720 microversion: 2.20 time: 0.325621 Oct 28 09:51:44.149238 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 464/876] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:43 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1720 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:45.181835 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7c29b4f7-bda6-46b4-8c10-7a7ed443b45e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:45.182630 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7c29b4f7-bda6-46b4-8c10-7a7ed443b45e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:45.185206 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=82 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:45.192061 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7c29b4f7-bda6-46b4-8c10-7a7ed443b45e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:45.192631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7c29b4f7-bda6-46b4-8c10-7a7ed443b45e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:45.193075 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7c29b4f7-bda6-46b4-8c10-7a7ed443b45e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:45.193805 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7c29b4f7-bda6-46b4-8c10-7a7ed443b45e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:45.194338 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7c29b4f7-bda6-46b4-8c10-7a7ed443b45e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:45.194752 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7c29b4f7-bda6-46b4-8c10-7a7ed443b45e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:45.342803 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7c29b4f7-bda6-46b4-8c10-7a7ed443b45e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:45.351990 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7c29b4f7-bda6-46b4-8c10-7a7ed443b45e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:45.353493 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7c29b4f7-bda6-46b4-8c10-7a7ed443b45e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:45.354789 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7c29b4f7-bda6-46b4-8c10-7a7ed443b45e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1720 microversion: 2.20 time: 0.177379 Oct 28 09:51:45.355385 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 465/877] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:45 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1720 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:46.382223 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-544701c4-f8e4-4883-9578-148a10d0bb21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:46.383053 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-544701c4-f8e4-4883-9578-148a10d0bb21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:46.390074 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-544701c4-f8e4-4883-9578-148a10d0bb21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:46.390594 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-544701c4-f8e4-4883-9578-148a10d0bb21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:46.391028 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-544701c4-f8e4-4883-9578-148a10d0bb21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:46.391741 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-544701c4-f8e4-4883-9578-148a10d0bb21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:46.392191 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-544701c4-f8e4-4883-9578-148a10d0bb21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:46.392608 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-544701c4-f8e4-4883-9578-148a10d0bb21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:46.543347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-544701c4-f8e4-4883-9578-148a10d0bb21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:46.546736 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:46.555542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-544701c4-f8e4-4883-9578-148a10d0bb21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:46.557359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-544701c4-f8e4-4883-9578-148a10d0bb21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:46.558820 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-544701c4-f8e4-4883-9578-148a10d0bb21 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1720 microversion: 2.20 time: 0.180783 Oct 28 09:51:46.559721 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 466/878] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:46 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1720 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:47.496433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-caf95281-25cb-47da-8d5a-e0c795d2d151 req-65e9e1ba-2d15-4b97-9860-5fd541113847 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b68e375d-f7ab-4d35-a8c7-081596d0e6e2", "name": "network-vif-unplugged", "status": "completed", "tag": "533de8d8-0438-4dd2-946d-f12312377670"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:47.531367 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-caf95281-25cb-47da-8d5a-e0c795d2d151 req-65e9e1ba-2d15-4b97-9860-5fd541113847 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:47.532031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-caf95281-25cb-47da-8d5a-e0c795d2d151 req-65e9e1ba-2d15-4b97-9860-5fd541113847 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:47.532771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-caf95281-25cb-47da-8d5a-e0c795d2d151 req-65e9e1ba-2d15-4b97-9860-5fd541113847 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:47.555883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-caf95281-25cb-47da-8d5a-e0c795d2d151 req-65e9e1ba-2d15-4b97-9860-5fd541113847 service nova] Creating event network-vif-unplugged:533de8d8-0438-4dd2-946d-f12312377670 for instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 on hv-cinder-zed-3 Oct 28 09:51:47.568639 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-caf95281-25cb-47da-8d5a-e0c795d2d151 req-65e9e1ba-2d15-4b97-9860-5fd541113847 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.077394 Oct 28 09:51:47.570854 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 467/879] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:51:47 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:51:47.597755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-95c845a4-3850-4951-b09a-e18727418ec9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:47.599116 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-95c845a4-3850-4951-b09a-e18727418ec9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:47.611258 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95c845a4-3850-4951-b09a-e18727418ec9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:47.612177 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95c845a4-3850-4951-b09a-e18727418ec9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:47.612977 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95c845a4-3850-4951-b09a-e18727418ec9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:47.614341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95c845a4-3850-4951-b09a-e18727418ec9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:47.615228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95c845a4-3850-4951-b09a-e18727418ec9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:47.615995 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-95c845a4-3850-4951-b09a-e18727418ec9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:48.100049 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-95c845a4-3850-4951-b09a-e18727418ec9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:48.109792 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-95c845a4-3850-4951-b09a-e18727418ec9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:48.111300 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-95c845a4-3850-4951-b09a-e18727418ec9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:48.112530 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-95c845a4-3850-4951-b09a-e18727418ec9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1720 microversion: 2.20 time: 0.526803 Oct 28 09:51:48.113102 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 468/880] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:47 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1720 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:51:49.140427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d0a80d39-40a2-4528-a3c7-5f61649dae13 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:49.141814 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d0a80d39-40a2-4528-a3c7-5f61649dae13 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:49.149250 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0a80d39-40a2-4528-a3c7-5f61649dae13 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:49.149723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0a80d39-40a2-4528-a3c7-5f61649dae13 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:49.150485 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0a80d39-40a2-4528-a3c7-5f61649dae13 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:49.150940 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0a80d39-40a2-4528-a3c7-5f61649dae13 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:49.151366 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0a80d39-40a2-4528-a3c7-5f61649dae13 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:49.151794 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d0a80d39-40a2-4528-a3c7-5f61649dae13 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:49.331109 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d0a80d39-40a2-4528-a3c7-5f61649dae13 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:49.345760 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d0a80d39-40a2-4528-a3c7-5f61649dae13 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:49.348496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d0a80d39-40a2-4528-a3c7-5f61649dae13 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:49.350573 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d0a80d39-40a2-4528-a3c7-5f61649dae13 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1663 microversion: 2.20 time: 0.214445 Oct 28 09:51:49.351642 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 469/881] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:49 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1663 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:49.360366 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f39aae92-5752-456a-a66d-7730621448cc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:49.363816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f39aae92-5752-456a-a66d-7730621448cc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:49.375250 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f39aae92-5752-456a-a66d-7730621448cc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:49.376101 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f39aae92-5752-456a-a66d-7730621448cc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:49.376834 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f39aae92-5752-456a-a66d-7730621448cc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:49.378008 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f39aae92-5752-456a-a66d-7730621448cc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:49.378573 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f39aae92-5752-456a-a66d-7730621448cc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:49.379335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f39aae92-5752-456a-a66d-7730621448cc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:49.435463 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f39aae92-5752-456a-a66d-7730621448cc tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.081904 Oct 28 09:51:49.437020 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 470/882] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:51:49 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments => generated 371 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:51:49.446744 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f64c53a7-0f40-4411-b995-0ae5c13724b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "1e760662-fa68-4777-adbf-785b1b2a9867"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:49.450976 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f64c53a7-0f40-4411-b995-0ae5c13724b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:49.462732 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f64c53a7-0f40-4411-b995-0ae5c13724b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:49.463553 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f64c53a7-0f40-4411-b995-0ae5c13724b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:49.464295 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f64c53a7-0f40-4411-b995-0ae5c13724b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:49.465596 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f64c53a7-0f40-4411-b995-0ae5c13724b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:49.466332 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f64c53a7-0f40-4411-b995-0ae5c13724b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:49.467140 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f64c53a7-0f40-4411-b995-0ae5c13724b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:49.820994 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f64c53a7-0f40-4411-b995-0ae5c13724b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "POST /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.382282 Oct 28 09:51:49.821799 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 471/883] 10.222.0.25 () {62 vars in 1364 bytes} [Fri Oct 28 09:51:49 2022] POST /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/os-volume_attachments => generated 188 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 09:51:49.827479 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6512fad1-4a2a-4485-a3c1-c69c3876c2a7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:49.828160 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6512fad1-4a2a-4485-a3c1-c69c3876c2a7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:49.836436 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6512fad1-4a2a-4485-a3c1-c69c3876c2a7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:49.836999 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6512fad1-4a2a-4485-a3c1-c69c3876c2a7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:49.837561 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6512fad1-4a2a-4485-a3c1-c69c3876c2a7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:49.838330 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6512fad1-4a2a-4485-a3c1-c69c3876c2a7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:49.838990 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6512fad1-4a2a-4485-a3c1-c69c3876c2a7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:49.839429 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6512fad1-4a2a-4485-a3c1-c69c3876c2a7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:49.884736 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6512fad1-4a2a-4485-a3c1-c69c3876c2a7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments/ba5635e0-5253-46eb-a1c7-a099b5af6518" status: 200 len: 194 microversion: 2.1 time: 0.061611 Oct 28 09:51:49.885430 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 472/884] 10.222.0.25 () {58 vars in 1413 bytes} [Fri Oct 28 09:51:49 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments/ba5635e0-5253-46eb-a1c7-a099b5af6518 => generated 194 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:51:49.904751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3621402a-6091-4565-99c2-72b1f5b285c7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:49.905486 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3621402a-6091-4565-99c2-72b1f5b285c7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:49.916691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3621402a-6091-4565-99c2-72b1f5b285c7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:49.917574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3621402a-6091-4565-99c2-72b1f5b285c7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:49.918347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3621402a-6091-4565-99c2-72b1f5b285c7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:49.919689 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3621402a-6091-4565-99c2-72b1f5b285c7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:49.920541 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3621402a-6091-4565-99c2-72b1f5b285c7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:49.922064 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3621402a-6091-4565-99c2-72b1f5b285c7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:50.043460 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-3621402a-6091-4565-99c2-72b1f5b285c7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lazy-loading 'services' on Instance uuid c28f1b03-788f-42f6-8577-9c81bffc41ae {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:51:50.137566 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3621402a-6091-4565-99c2-72b1f5b285c7 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments/ba5635e0-5253-46eb-a1c7-a099b5af6518" status: 202 len: 0 microversion: 2.1 time: 0.239730 Oct 28 09:51:50.143659 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 473/885] 10.222.0.25 () {58 vars in 1416 bytes} [Fri Oct 28 09:51:49 2022] DELETE /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments/ba5635e0-5253-46eb-a1c7-a099b5af6518 => generated 0 bytes in 243 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:51:50.148672 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-da57fecd-7508-42eb-a134-dc0fc321037c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:50.150107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-da57fecd-7508-42eb-a134-dc0fc321037c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:50.165504 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-da57fecd-7508-42eb-a134-dc0fc321037c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:50.166542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-da57fecd-7508-42eb-a134-dc0fc321037c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:50.167376 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-da57fecd-7508-42eb-a134-dc0fc321037c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:50.168637 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-da57fecd-7508-42eb-a134-dc0fc321037c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:50.169477 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-da57fecd-7508-42eb-a134-dc0fc321037c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:50.171751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-da57fecd-7508-42eb-a134-dc0fc321037c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:50.502617 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-da57fecd-7508-42eb-a134-dc0fc321037c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "DELETE /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/os-volume_attachments/1e760662-fa68-4777-adbf-785b1b2a9867" status: 202 len: 0 microversion: 2.20 time: 0.361669 Oct 28 09:51:50.503205 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 474/886] 10.222.0.25 () {60 vars in 1457 bytes} [Fri Oct 28 09:51:50 2022] DELETE /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/os-volume_attachments/1e760662-fa68-4777-adbf-785b1b2a9867 => generated 0 bytes in 364 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:51:50.577873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6d4b11ba-6f41-4e26-b7b4-c6d0d89fb803 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:51:50.578510 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6d4b11ba-6f41-4e26-b7b4-c6d0d89fb803 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:50.585206 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d4b11ba-6f41-4e26-b7b4-c6d0d89fb803 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:50.585793 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d4b11ba-6f41-4e26-b7b4-c6d0d89fb803 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:50.586260 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d4b11ba-6f41-4e26-b7b4-c6d0d89fb803 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:50.587086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d4b11ba-6f41-4e26-b7b4-c6d0d89fb803 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:50.587623 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d4b11ba-6f41-4e26-b7b4-c6d0d89fb803 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:50.588124 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6d4b11ba-6f41-4e26-b7b4-c6d0d89fb803 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:50.625285 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6d4b11ba-6f41-4e26-b7b4-c6d0d89fb803 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=127030) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 28 09:51:50.702005 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6d4b11ba-6f41-4e26-b7b4-c6d0d89fb803 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "POST /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/action" status: 202 len: 0 microversion: 2.20 time: 0.128794 Oct 28 09:51:50.703402 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 475/887] 10.222.0.25 () {62 vars in 1319 bytes} [Fri Oct 28 09:51:50 2022] POST /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/action => generated 0 bytes in 131 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 28 09:51:50.711767 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a6cf0508-1d2f-45a4-89ff-8cf3b8f7f976 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:50.712445 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a6cf0508-1d2f-45a4-89ff-8cf3b8f7f976 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:50.719187 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6cf0508-1d2f-45a4-89ff-8cf3b8f7f976 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:50.719826 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6cf0508-1d2f-45a4-89ff-8cf3b8f7f976 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:50.720349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6cf0508-1d2f-45a4-89ff-8cf3b8f7f976 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:50.721150 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6cf0508-1d2f-45a4-89ff-8cf3b8f7f976 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:50.721645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6cf0508-1d2f-45a4-89ff-8cf3b8f7f976 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:50.722373 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a6cf0508-1d2f-45a4-89ff-8cf3b8f7f976 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:50.892267 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a6cf0508-1d2f-45a4-89ff-8cf3b8f7f976 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:50.902113 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a6cf0508-1d2f-45a4-89ff-8cf3b8f7f976 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:50.904206 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a6cf0508-1d2f-45a4-89ff-8cf3b8f7f976 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:50.905460 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a6cf0508-1d2f-45a4-89ff-8cf3b8f7f976 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1671 microversion: 2.20 time: 0.197989 Oct 28 09:51:50.906035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 476/888] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:50 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1671 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:51.932041 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5f9b75e8-a713-48af-8e20-d8b7c6ae281c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:51.933103 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5f9b75e8-a713-48af-8e20-d8b7c6ae281c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:51.940094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f9b75e8-a713-48af-8e20-d8b7c6ae281c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:51.940633 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f9b75e8-a713-48af-8e20-d8b7c6ae281c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:51.941111 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f9b75e8-a713-48af-8e20-d8b7c6ae281c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:51.941865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f9b75e8-a713-48af-8e20-d8b7c6ae281c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:51.942368 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f9b75e8-a713-48af-8e20-d8b7c6ae281c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:51.942893 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f9b75e8-a713-48af-8e20-d8b7c6ae281c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:51.947670 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=84,nova_cell1:INSERT=7,nova_cell1:UPDATE=9 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:52.095601 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5f9b75e8-a713-48af-8e20-d8b7c6ae281c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:52.105740 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5f9b75e8-a713-48af-8e20-d8b7c6ae281c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:52.107426 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5f9b75e8-a713-48af-8e20-d8b7c6ae281c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:52.108706 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5f9b75e8-a713-48af-8e20-d8b7c6ae281c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1671 microversion: 2.20 time: 0.180787 Oct 28 09:51:52.109356 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 477/889] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:51 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1671 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:53.135116 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d5081862-26a7-45db-aa83-09de1ca74cf8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:53.135806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d5081862-26a7-45db-aa83-09de1ca74cf8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:53.143347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d5081862-26a7-45db-aa83-09de1ca74cf8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:53.143919 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d5081862-26a7-45db-aa83-09de1ca74cf8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:53.144357 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d5081862-26a7-45db-aa83-09de1ca74cf8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:53.145061 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d5081862-26a7-45db-aa83-09de1ca74cf8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:53.145505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d5081862-26a7-45db-aa83-09de1ca74cf8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:53.145919 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d5081862-26a7-45db-aa83-09de1ca74cf8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:53.339890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d5081862-26a7-45db-aa83-09de1ca74cf8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:53.364901 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d5081862-26a7-45db-aa83-09de1ca74cf8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:53.366488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d5081862-26a7-45db-aa83-09de1ca74cf8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:53.368607 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d5081862-26a7-45db-aa83-09de1ca74cf8 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1671 microversion: 2.20 time: 0.236737 Oct 28 09:51:53.369501 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 478/890] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:53 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1671 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:54.394906 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2b60d939-d5e5-49a1-876c-31bb0931b478 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:54.395939 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2b60d939-d5e5-49a1-876c-31bb0931b478 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:54.403133 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2b60d939-d5e5-49a1-876c-31bb0931b478 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:54.403991 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2b60d939-d5e5-49a1-876c-31bb0931b478 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:54.404520 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2b60d939-d5e5-49a1-876c-31bb0931b478 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:54.405348 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2b60d939-d5e5-49a1-876c-31bb0931b478 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:54.405911 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2b60d939-d5e5-49a1-876c-31bb0931b478 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:54.406482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2b60d939-d5e5-49a1-876c-31bb0931b478 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:54.596226 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2b60d939-d5e5-49a1-876c-31bb0931b478 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:54.630559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2b60d939-d5e5-49a1-876c-31bb0931b478 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:54.634908 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2b60d939-d5e5-49a1-876c-31bb0931b478 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:54.639302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2b60d939-d5e5-49a1-876c-31bb0931b478 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1671 microversion: 2.20 time: 0.248167 Oct 28 09:51:54.640973 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 479/891] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:54 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1671 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:55.689805 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e35784a4-dcc9-4fed-b6f7-ce0f0e17399d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:55.690607 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e35784a4-dcc9-4fed-b6f7-ce0f0e17399d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:55.697814 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e35784a4-dcc9-4fed-b6f7-ce0f0e17399d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:55.698312 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e35784a4-dcc9-4fed-b6f7-ce0f0e17399d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:55.698786 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e35784a4-dcc9-4fed-b6f7-ce0f0e17399d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:55.699544 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e35784a4-dcc9-4fed-b6f7-ce0f0e17399d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:55.699985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e35784a4-dcc9-4fed-b6f7-ce0f0e17399d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:55.700415 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e35784a4-dcc9-4fed-b6f7-ce0f0e17399d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:55.965113 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e35784a4-dcc9-4fed-b6f7-ce0f0e17399d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:55.981274 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e35784a4-dcc9-4fed-b6f7-ce0f0e17399d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:55.985998 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e35784a4-dcc9-4fed-b6f7-ce0f0e17399d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:55.990732 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e35784a4-dcc9-4fed-b6f7-ce0f0e17399d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1671 microversion: 2.20 time: 0.304904 Oct 28 09:51:55.992833 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 480/892] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:55 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1671 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:57.018460 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1831423b-90d2-43cc-8284-6bbcf9c4d144 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:57.019575 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1831423b-90d2-43cc-8284-6bbcf9c4d144 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:57.035020 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1831423b-90d2-43cc-8284-6bbcf9c4d144 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:57.035519 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1831423b-90d2-43cc-8284-6bbcf9c4d144 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:57.036150 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1831423b-90d2-43cc-8284-6bbcf9c4d144 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:57.036951 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1831423b-90d2-43cc-8284-6bbcf9c4d144 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:57.038261 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1831423b-90d2-43cc-8284-6bbcf9c4d144 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:57.038728 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1831423b-90d2-43cc-8284-6bbcf9c4d144 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:57.285631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1831423b-90d2-43cc-8284-6bbcf9c4d144 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:57.302496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1831423b-90d2-43cc-8284-6bbcf9c4d144 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:57.305737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1831423b-90d2-43cc-8284-6bbcf9c4d144 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:57.308287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1831423b-90d2-43cc-8284-6bbcf9c4d144 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1671 microversion: 2.20 time: 0.294350 Oct 28 09:51:57.310774 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 481/893] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:57 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1671 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:58.350587 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-66c976d8-0a87-42cb-91c6-fd59261c5033 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:58.351673 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-66c976d8-0a87-42cb-91c6-fd59261c5033 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:58.361451 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-66c976d8-0a87-42cb-91c6-fd59261c5033 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:58.361451 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-66c976d8-0a87-42cb-91c6-fd59261c5033 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:58.361765 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-66c976d8-0a87-42cb-91c6-fd59261c5033 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:58.363260 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-66c976d8-0a87-42cb-91c6-fd59261c5033 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:58.363958 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-66c976d8-0a87-42cb-91c6-fd59261c5033 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:58.365314 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-66c976d8-0a87-42cb-91c6-fd59261c5033 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:58.588239 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-66c976d8-0a87-42cb-91c6-fd59261c5033 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:58.602341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-66c976d8-0a87-42cb-91c6-fd59261c5033 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:58.604690 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-66c976d8-0a87-42cb-91c6-fd59261c5033 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:58.606187 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-66c976d8-0a87-42cb-91c6-fd59261c5033 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1673 microversion: 2.20 time: 0.260111 Oct 28 09:51:58.606996 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 482/894] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:58 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1673 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:51:59.079864 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f414032e-d90b-48b2-817d-de8e551d878f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:59.080378 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f414032e-d90b-48b2-817d-de8e551d878f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:59.088295 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f414032e-d90b-48b2-817d-de8e551d878f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:59.088487 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f414032e-d90b-48b2-817d-de8e551d878f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:59.089059 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f414032e-d90b-48b2-817d-de8e551d878f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:59.089915 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f414032e-d90b-48b2-817d-de8e551d878f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:59.090668 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f414032e-d90b-48b2-817d-de8e551d878f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:59.091096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f414032e-d90b-48b2-817d-de8e551d878f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:59.156130 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f414032e-d90b-48b2-817d-de8e551d878f tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments/873eedf8-2b3f-4477-b37e-85d67d58c2ff" status: 200 len: 194 microversion: 2.1 time: 0.079861 Oct 28 09:51:59.163692 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 483/895] 10.222.0.25 () {58 vars in 1413 bytes} [Fri Oct 28 09:51:59 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments/873eedf8-2b3f-4477-b37e-85d67d58c2ff => generated 194 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:51:59.174668 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-786ccf60-0079-4d82-b2ea-b5c487a6fb91 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:59.175375 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-786ccf60-0079-4d82-b2ea-b5c487a6fb91 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:59.188188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-786ccf60-0079-4d82-b2ea-b5c487a6fb91 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:59.188982 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-786ccf60-0079-4d82-b2ea-b5c487a6fb91 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:59.189629 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-786ccf60-0079-4d82-b2ea-b5c487a6fb91 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:59.190928 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-786ccf60-0079-4d82-b2ea-b5c487a6fb91 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:59.191409 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-786ccf60-0079-4d82-b2ea-b5c487a6fb91 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:59.191837 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-786ccf60-0079-4d82-b2ea-b5c487a6fb91 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:59.309834 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-786ccf60-0079-4d82-b2ea-b5c487a6fb91 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lazy-loading 'services' on Instance uuid c28f1b03-788f-42f6-8577-9c81bffc41ae {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:51:59.474137 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-786ccf60-0079-4d82-b2ea-b5c487a6fb91 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments/873eedf8-2b3f-4477-b37e-85d67d58c2ff" status: 202 len: 0 microversion: 2.1 time: 0.305707 Oct 28 09:51:59.475260 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 484/896] 10.222.0.25 () {58 vars in 1416 bytes} [Fri Oct 28 09:51:59 2022] DELETE /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments/873eedf8-2b3f-4477-b37e-85d67d58c2ff => generated 0 bytes in 308 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:51:59.637673 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6168684d-7697-4b65-80ab-5660a605e1ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:51:59.638661 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6168684d-7697-4b65-80ab-5660a605e1ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:51:59.653871 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6168684d-7697-4b65-80ab-5660a605e1ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:59.657176 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6168684d-7697-4b65-80ab-5660a605e1ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:59.657676 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6168684d-7697-4b65-80ab-5660a605e1ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:59.659709 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6168684d-7697-4b65-80ab-5660a605e1ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:51:59.660257 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6168684d-7697-4b65-80ab-5660a605e1ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:51:59.664185 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6168684d-7697-4b65-80ab-5660a605e1ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:51:59.879472 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6168684d-7697-4b65-80ab-5660a605e1ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:59.898261 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6168684d-7697-4b65-80ab-5660a605e1ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:59.901401 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6168684d-7697-4b65-80ab-5660a605e1ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:51:59.904302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6168684d-7697-4b65-80ab-5660a605e1ca tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 0.271143 Oct 28 09:51:59.906449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 485/897] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:51:59 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:52:00.942887 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-76f8e2f7-541d-406a-92cb-dd7d705fa5b0 tempest-TestVolumeBootPattern-1984064024 tempest-TestVolumeBootPattern-1984064024-project] Calling method '.version_select at 0x7f271a322280>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:01.011555 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-76f8e2f7-541d-406a-92cb-dd7d705fa5b0 tempest-TestVolumeBootPattern-1984064024 tempest-TestVolumeBootPattern-1984064024-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-196867844" status: 202 len: 0 microversion: 2.1 time: 0.086204 Oct 28 09:52:01.012458 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 486/898] 10.222.0.25 () {58 vars in 1260 bytes} [Fri Oct 28 09:52:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-196867844 => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:52:01.022419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e09fe343-204f-4d00-a065-e43db1300451 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:01.027412 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e09fe343-204f-4d00-a065-e43db1300451 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:01.042038 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e09fe343-204f-4d00-a065-e43db1300451 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:01.042574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e09fe343-204f-4d00-a065-e43db1300451 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:01.043128 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e09fe343-204f-4d00-a065-e43db1300451 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:01.043748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e09fe343-204f-4d00-a065-e43db1300451 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:01.044213 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e09fe343-204f-4d00-a065-e43db1300451 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:01.044663 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e09fe343-204f-4d00-a065-e43db1300451 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:01.592161 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e09fe343-204f-4d00-a065-e43db1300451 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:01.623760 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e09fe343-204f-4d00-a065-e43db1300451 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:01.627235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e09fe343-204f-4d00-a065-e43db1300451 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:01.629120 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e09fe343-204f-4d00-a065-e43db1300451 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 0.615403 Oct 28 09:52:01.635159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 487/899] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:01 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:52:01.652960 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ad24c55b-246c-4f27-a764-25d130a71b6d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:01.660815 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ad24c55b-246c-4f27-a764-25d130a71b6d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:01.685918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad24c55b-246c-4f27-a764-25d130a71b6d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:01.687498 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad24c55b-246c-4f27-a764-25d130a71b6d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:01.688580 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad24c55b-246c-4f27-a764-25d130a71b6d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:01.690164 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad24c55b-246c-4f27-a764-25d130a71b6d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:01.691503 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad24c55b-246c-4f27-a764-25d130a71b6d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:01.692693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad24c55b-246c-4f27-a764-25d130a71b6d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:01.800517 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ad24c55b-246c-4f27-a764-25d130a71b6d tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.161054 Oct 28 09:52:01.804278 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 488/900] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:52:01 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments => generated 25 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 09:52:02.241199 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-83e759d4-6fff-49ce-983f-d906cb621262 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:02.243676 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-83e759d4-6fff-49ce-983f-d906cb621262 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:02.274481 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83e759d4-6fff-49ce-983f-d906cb621262 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:02.284850 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83e759d4-6fff-49ce-983f-d906cb621262 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:02.286013 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83e759d4-6fff-49ce-983f-d906cb621262 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:02.299079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83e759d4-6fff-49ce-983f-d906cb621262 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:02.299822 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83e759d4-6fff-49ce-983f-d906cb621262 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:02.302322 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83e759d4-6fff-49ce-983f-d906cb621262 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:02.427204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-83e759d4-6fff-49ce-983f-d906cb621262 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.190434 Oct 28 09:52:02.431964 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 489/901] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:52:02 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae/os-volume_attachments => generated 25 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 09:52:02.662608 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-996507c7-ebb1-4ad6-8814-de203e5aa12a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:02.663584 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-996507c7-ebb1-4ad6-8814-de203e5aa12a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:02.684646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-996507c7-ebb1-4ad6-8814-de203e5aa12a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:02.686104 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-996507c7-ebb1-4ad6-8814-de203e5aa12a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:02.688923 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-996507c7-ebb1-4ad6-8814-de203e5aa12a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:02.692943 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-996507c7-ebb1-4ad6-8814-de203e5aa12a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:02.696593 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-996507c7-ebb1-4ad6-8814-de203e5aa12a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:02.699323 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-996507c7-ebb1-4ad6-8814-de203e5aa12a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:04.533853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-996507c7-ebb1-4ad6-8814-de203e5aa12a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:04.559066 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-996507c7-ebb1-4ad6-8814-de203e5aa12a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:04.570551 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-996507c7-ebb1-4ad6-8814-de203e5aa12a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:04.573580 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-996507c7-ebb1-4ad6-8814-de203e5aa12a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 1.920080 Oct 28 09:52:04.582135 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 490/902] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:02 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 1930 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:52:04.616039 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2802d7f6-eba6-4559-87d1-3f9c2a6f9ef1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:04.620058 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2802d7f6-eba6-4559-87d1-3f9c2a6f9ef1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:04.641137 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2802d7f6-eba6-4559-87d1-3f9c2a6f9ef1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:04.642179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2802d7f6-eba6-4559-87d1-3f9c2a6f9ef1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:04.643197 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2802d7f6-eba6-4559-87d1-3f9c2a6f9ef1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:04.644157 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2802d7f6-eba6-4559-87d1-3f9c2a6f9ef1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:04.645112 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2802d7f6-eba6-4559-87d1-3f9c2a6f9ef1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:04.645846 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2802d7f6-eba6-4559-87d1-3f9c2a6f9ef1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:04.731658 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2802d7f6-eba6-4559-87d1-3f9c2a6f9ef1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Going to try to terminate instance {{(pid=127030) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:52:04.992230 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2802d7f6-eba6-4559-87d1-3f9c2a6f9ef1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 204 len: 0 microversion: 2.1 time: 0.408345 Oct 28 09:52:05.008226 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 491/903] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:52:04 2022] DELETE /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 0 bytes in 422 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:52:05.017395 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-821801e5-e839-45ac-81e9-94b5f8e04677 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:05.018335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-821801e5-e839-45ac-81e9-94b5f8e04677 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:05.052725 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-821801e5-e839-45ac-81e9-94b5f8e04677 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:05.053289 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-821801e5-e839-45ac-81e9-94b5f8e04677 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:05.053875 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-821801e5-e839-45ac-81e9-94b5f8e04677 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:05.068538 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-821801e5-e839-45ac-81e9-94b5f8e04677 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:05.069142 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-821801e5-e839-45ac-81e9-94b5f8e04677 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:05.071276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-821801e5-e839-45ac-81e9-94b5f8e04677 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:05.157198 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:05.600826 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-821801e5-e839-45ac-81e9-94b5f8e04677 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:05.610244 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-821801e5-e839-45ac-81e9-94b5f8e04677 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1675 microversion: 2.1 time: 0.599991 Oct 28 09:52:05.614376 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 492/904] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:05 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1675 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:52:05.627574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-51152819-7249-471e-bfe4-259da995afda tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:05.628010 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-51152819-7249-471e-bfe4-259da995afda tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:05.648434 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-51152819-7249-471e-bfe4-259da995afda tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:05.648998 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-51152819-7249-471e-bfe4-259da995afda tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:05.649446 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-51152819-7249-471e-bfe4-259da995afda tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:05.650182 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-51152819-7249-471e-bfe4-259da995afda tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:05.651897 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-51152819-7249-471e-bfe4-259da995afda tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:05.654234 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-51152819-7249-471e-bfe4-259da995afda tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:05.698096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-51152819-7249-471e-bfe4-259da995afda tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/ips" status: 200 len: 151 microversion: 2.1 time: 0.082551 Oct 28 09:52:05.698717 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 493/905] 10.222.0.25 () {58 vars in 1248 bytes} [Fri Oct 28 09:52:05 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/ips => generated 151 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:52:05.698717 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f6f1d0fd-8d4a-4450-a3af-519c2ff694b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:05.698717 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f6f1d0fd-8d4a-4450-a3af-519c2ff694b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:05.728082 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6f1d0fd-8d4a-4450-a3af-519c2ff694b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:05.728567 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6f1d0fd-8d4a-4450-a3af-519c2ff694b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:05.736588 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6f1d0fd-8d4a-4450-a3af-519c2ff694b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:05.739862 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6f1d0fd-8d4a-4450-a3af-519c2ff694b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:05.741265 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6f1d0fd-8d4a-4450-a3af-519c2ff694b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:05.742988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f6f1d0fd-8d4a-4450-a3af-519c2ff694b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:06.368163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f6f1d0fd-8d4a-4450-a3af-519c2ff694b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:06.386866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f6f1d0fd-8d4a-4450-a3af-519c2ff694b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:06.405005 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f6f1d0fd-8d4a-4450-a3af-519c2ff694b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:06.407904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f6f1d0fd-8d4a-4450-a3af-519c2ff694b1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 0.707770 Oct 28 09:52:06.435094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 494/906] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:05 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 736 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:52:06.655718 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-54521f48-3940-4367-98ca-10145ded7ac1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:06.659858 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-54521f48-3940-4367-98ca-10145ded7ac1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:06.683623 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-54521f48-3940-4367-98ca-10145ded7ac1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:06.684802 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-54521f48-3940-4367-98ca-10145ded7ac1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:06.685639 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-54521f48-3940-4367-98ca-10145ded7ac1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:06.687755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-54521f48-3940-4367-98ca-10145ded7ac1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:06.689042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-54521f48-3940-4367-98ca-10145ded7ac1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:06.689967 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-54521f48-3940-4367-98ca-10145ded7ac1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:07.380986 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-54521f48-3940-4367-98ca-10145ded7ac1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:07.397335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-54521f48-3940-4367-98ca-10145ded7ac1 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1675 microversion: 2.1 time: 0.751456 Oct 28 09:52:07.416459 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 495/907] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:06 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1675 bytes in 771 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:52:07.474541 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8c28858f-44e2-45f9-8855-2c9d85f0ce9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:07.474846 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8c28858f-44e2-45f9-8855-2c9d85f0ce9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:07.518521 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8c28858f-44e2-45f9-8855-2c9d85f0ce9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:07.519416 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8c28858f-44e2-45f9-8855-2c9d85f0ce9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:07.525419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8c28858f-44e2-45f9-8855-2c9d85f0ce9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:07.528948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8c28858f-44e2-45f9-8855-2c9d85f0ce9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:07.529730 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8c28858f-44e2-45f9-8855-2c9d85f0ce9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:07.531220 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8c28858f-44e2-45f9-8855-2c9d85f0ce9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:08.648495 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8c28858f-44e2-45f9-8855-2c9d85f0ce9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:08.658704 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8c28858f-44e2-45f9-8855-2c9d85f0ce9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:08.666631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8c28858f-44e2-45f9-8855-2c9d85f0ce9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:08.668155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8c28858f-44e2-45f9-8855-2c9d85f0ce9e tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 1.209470 Oct 28 09:52:08.668784 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 496/908] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:07 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 1211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:52:08.693726 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7023e415-0752-4598-ab60-521134c963fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "aa4c72e7-5fe1-42bd-9a3b-ce956b1736c4"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:52:08.705505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7023e415-0752-4598-ab60-521134c963fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:08.728183 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7023e415-0752-4598-ab60-521134c963fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:08.728729 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7023e415-0752-4598-ab60-521134c963fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:08.729172 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7023e415-0752-4598-ab60-521134c963fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:08.740502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7023e415-0752-4598-ab60-521134c963fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:08.741049 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7023e415-0752-4598-ab60-521134c963fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:08.741763 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7023e415-0752-4598-ab60-521134c963fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:09.732376 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7023e415-0752-4598-ab60-521134c963fe tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "POST /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.062240 Oct 28 09:52:09.744771 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 497/909] 10.222.0.25 () {60 vars in 1323 bytes} [Fri Oct 28 09:52:08 2022] POST /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/os-volume_attachments => generated 194 bytes in 1075 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:52:09.760433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-31db8c81-b6f6-40b7-b749-356cdeb29a97 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:09.763932 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-31db8c81-b6f6-40b7-b749-356cdeb29a97 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:09.785001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-31db8c81-b6f6-40b7-b749-356cdeb29a97 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:09.785860 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-31db8c81-b6f6-40b7-b749-356cdeb29a97 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:09.791522 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-31db8c81-b6f6-40b7-b749-356cdeb29a97 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:09.792964 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-31db8c81-b6f6-40b7-b749-356cdeb29a97 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:09.793510 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-31db8c81-b6f6-40b7-b749-356cdeb29a97 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:09.794040 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-31db8c81-b6f6-40b7-b749-356cdeb29a97 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:12.800749 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-31db8c81-b6f6-40b7-b749-356cdeb29a97 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:12.829454 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-31db8c81-b6f6-40b7-b749-356cdeb29a97 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1675 microversion: 2.1 time: 3.082978 Oct 28 09:52:12.830339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 498/910] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:09 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1675 bytes in 3085 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:52:12.839064 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-bc4f7486-ee4c-4fec-981f-dd93b7bc1c49 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:12.839064 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-bc4f7486-ee4c-4fec-981f-dd93b7bc1c49 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:12.880810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bc4f7486-ee4c-4fec-981f-dd93b7bc1c49 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:12.881295 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bc4f7486-ee4c-4fec-981f-dd93b7bc1c49 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:12.885371 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bc4f7486-ee4c-4fec-981f-dd93b7bc1c49 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:12.887058 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bc4f7486-ee4c-4fec-981f-dd93b7bc1c49 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:12.895384 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bc4f7486-ee4c-4fec-981f-dd93b7bc1c49 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:12.896131 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bc4f7486-ee4c-4fec-981f-dd93b7bc1c49 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:14.201564 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-bc4f7486-ee4c-4fec-981f-dd93b7bc1c49 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:14.238766 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-bc4f7486-ee4c-4fec-981f-dd93b7bc1c49 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:14.252196 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-bc4f7486-ee4c-4fec-981f-dd93b7bc1c49 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:14.254710 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-bc4f7486-ee4c-4fec-981f-dd93b7bc1c49 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 1.423098 Oct 28 09:52:14.263539 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 499/911] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:12 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 1433 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:52:14.279710 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4e7149c1-99d3-4680-bd43-60e5d7935e01 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:14.280855 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4e7149c1-99d3-4680-bd43-60e5d7935e01 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:14.309453 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e7149c1-99d3-4680-bd43-60e5d7935e01 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:14.310304 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e7149c1-99d3-4680-bd43-60e5d7935e01 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:14.316753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e7149c1-99d3-4680-bd43-60e5d7935e01 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:14.328268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e7149c1-99d3-4680-bd43-60e5d7935e01 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:14.329265 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e7149c1-99d3-4680-bd43-60e5d7935e01 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:14.330064 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e7149c1-99d3-4680-bd43-60e5d7935e01 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:15.000796 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-4e7149c1-99d3-4680-bd43-60e5d7935e01 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:15.072740 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4e7149c1-99d3-4680-bd43-60e5d7935e01 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1607 microversion: 2.1 time: 0.805892 Oct 28 09:52:15.088395 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 500/912] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:14 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1607 bytes in 824 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:52:15.209863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-72dc2300-77a1-4757-bdcc-9380a7f88f1b req-6b6eaefb-e966-4c3f-aaba-0324469eb45f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c28f1b03-788f-42f6-8577-9c81bffc41ae", "name": "network-vif-deleted", "tag": "1fff934b-bb90-4e1c-b8bd-845cc947058d"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:52:15.244799 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=82,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:15.251963 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-72dc2300-77a1-4757-bdcc-9380a7f88f1b req-6b6eaefb-e966-4c3f-aaba-0324469eb45f service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:15.252448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-72dc2300-77a1-4757-bdcc-9380a7f88f1b req-6b6eaefb-e966-4c3f-aaba-0324469eb45f service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:15.252994 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-72dc2300-77a1-4757-bdcc-9380a7f88f1b req-6b6eaefb-e966-4c3f-aaba-0324469eb45f service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:15.297943 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-72dc2300-77a1-4757-bdcc-9380a7f88f1b req-6b6eaefb-e966-4c3f-aaba-0324469eb45f service nova] Creating event network-vif-deleted:1fff934b-bb90-4e1c-b8bd-845cc947058d for instance c28f1b03-788f-42f6-8577-9c81bffc41ae on hv-cinder-zed-3 Oct 28 09:52:15.315287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-72dc2300-77a1-4757-bdcc-9380a7f88f1b req-6b6eaefb-e966-4c3f-aaba-0324469eb45f service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.118270 Oct 28 09:52:15.322143 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 501/913] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:52:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:52:15.337903 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-415d04e3-f06c-4f28-9afd-0dbb988708c0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:15.339688 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-415d04e3-f06c-4f28-9afd-0dbb988708c0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:15.360482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-415d04e3-f06c-4f28-9afd-0dbb988708c0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:15.363262 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-415d04e3-f06c-4f28-9afd-0dbb988708c0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:15.364223 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-415d04e3-f06c-4f28-9afd-0dbb988708c0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:15.367938 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-415d04e3-f06c-4f28-9afd-0dbb988708c0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:15.369767 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-415d04e3-f06c-4f28-9afd-0dbb988708c0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:15.372461 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-415d04e3-f06c-4f28-9afd-0dbb988708c0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:16.019379 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-415d04e3-f06c-4f28-9afd-0dbb988708c0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:16.065681 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-415d04e3-f06c-4f28-9afd-0dbb988708c0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:16.075027 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-415d04e3-f06c-4f28-9afd-0dbb988708c0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:16.084367 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-415d04e3-f06c-4f28-9afd-0dbb988708c0 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 0.759862 Oct 28 09:52:16.085988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 502/914] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:15 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 763 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:52:16.111850 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5f47c402-6577-40cd-8957-8fb16d7387df tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:16.113217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5f47c402-6577-40cd-8957-8fb16d7387df tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:16.138766 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f47c402-6577-40cd-8957-8fb16d7387df tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:16.142648 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f47c402-6577-40cd-8957-8fb16d7387df tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:16.143887 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f47c402-6577-40cd-8957-8fb16d7387df tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:16.150248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f47c402-6577-40cd-8957-8fb16d7387df tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:16.156729 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f47c402-6577-40cd-8957-8fb16d7387df tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:16.159457 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5f47c402-6577-40cd-8957-8fb16d7387df tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:16.516201 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-5f47c402-6577-40cd-8957-8fb16d7387df tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lazy-loading 'services' on Instance uuid 9d28bd27-0d1c-45d2-8d9d-81595af44f70 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:52:16.716752 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5f47c402-6577-40cd-8957-8fb16d7387df tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/os-volume_attachments/aa4c72e7-5fe1-42bd-9a3b-ce956b1736c4" status: 202 len: 0 microversion: 2.1 time: 0.628077 Oct 28 09:52:16.725876 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 503/915] 10.222.0.25 () {58 vars in 1416 bytes} [Fri Oct 28 09:52:16 2022] DELETE /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/os-volume_attachments/aa4c72e7-5fe1-42bd-9a3b-ce956b1736c4 => generated 0 bytes in 639 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:52:16.732313 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:16.735671 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:16.755655 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:16.759088 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:16.759742 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:16.763591 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:16.766212 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:16.766883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:16.831918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:16.833010 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:16.834878 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:16.835583 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lazy-loading 'fault' on Instance uuid c28f1b03-788f-42f6-8577-9c81bffc41ae {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:52:16.903457 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4237ca76acd4f7483aaf66e1f5d2eea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2fcce993e6b4388bde493f0702fd49d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:16.916349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7e416275-6f8d-40d6-a42d-61bc9a6cf080 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 200 len: 1328 microversion: 2.1 time: 0.188954 Oct 28 09:52:16.917900 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 504/916] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:16 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 1328 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:52:16.928323 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8d8c52b8-54dd-415d-8b09-7deaa07c33c2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:16.935617 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8d8c52b8-54dd-415d-8b09-7deaa07c33c2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:16.945743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d8c52b8-54dd-415d-8b09-7deaa07c33c2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:16.946720 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d8c52b8-54dd-415d-8b09-7deaa07c33c2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:16.947341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d8c52b8-54dd-415d-8b09-7deaa07c33c2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:16.949248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d8c52b8-54dd-415d-8b09-7deaa07c33c2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:16.949965 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d8c52b8-54dd-415d-8b09-7deaa07c33c2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:16.951352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8d8c52b8-54dd-415d-8b09-7deaa07c33c2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:17.044312 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8d8c52b8-54dd-415d-8b09-7deaa07c33c2 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.124038 Oct 28 09:52:17.051714 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 505/917] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:52:16 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/os-volume_attachments => generated 197 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:52:17.126759 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-69393249-1891-41ba-9aee-4c1c79161c3c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:17.127687 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-69393249-1891-41ba-9aee-4c1c79161c3c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:17.150346 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69393249-1891-41ba-9aee-4c1c79161c3c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:17.151211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69393249-1891-41ba-9aee-4c1c79161c3c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:17.151845 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69393249-1891-41ba-9aee-4c1c79161c3c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:17.152899 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69393249-1891-41ba-9aee-4c1c79161c3c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:17.153568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69393249-1891-41ba-9aee-4c1c79161c3c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:17.154420 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69393249-1891-41ba-9aee-4c1c79161c3c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:17.450976 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-69393249-1891-41ba-9aee-4c1c79161c3c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:17.464128 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-69393249-1891-41ba-9aee-4c1c79161c3c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:17.467332 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-69393249-1891-41ba-9aee-4c1c79161c3c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:17.468873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-69393249-1891-41ba-9aee-4c1c79161c3c tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 0.349186 Oct 28 09:52:17.494791 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 506/918] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:17 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:52:17.966377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-cbb27c05-8ce8-4843-99a5-c855cd60c486 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:17.968527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-cbb27c05-8ce8-4843-99a5-c855cd60c486 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:17.980854 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbb27c05-8ce8-4843-99a5-c855cd60c486 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:17.985346 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbb27c05-8ce8-4843-99a5-c855cd60c486 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:17.986343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbb27c05-8ce8-4843-99a5-c855cd60c486 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:17.987471 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbb27c05-8ce8-4843-99a5-c855cd60c486 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:17.988774 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbb27c05-8ce8-4843-99a5-c855cd60c486 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:17.991835 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbb27c05-8ce8-4843-99a5-c855cd60c486 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:18.029141 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-cbb27c05-8ce8-4843-99a5-c855cd60c486 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] HTTP exception thrown: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-cbb27c05-8ce8-4843-99a5-c855cd60c486 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Request method failure captured: Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae HTTP/1.1 Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sr9rY_ITV7HhEadU-8kVwknlsO0ILZxCejbC0tv-IXJnixigZ0yNp9FPJ0cREY_BiaBySs0qEXDy-B2HhLgm_7R6Uo426RjnExTKsP2_sj58xycvGdyeCTUklJQqy6gTOR8ZMSLaSiUs2I6gLtF7HzqTatq6lKFNab2p6V5DQOaCQ Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: f2fcce993e6b4388bde493f0702fd49d Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:18.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:18.035631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f2fcce993e6b4388bde493f0702fd49d"}]}] Oct 28 09:52:18.035631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:18.035631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: f2fcce993e6b4388bde493f0702fd49d Oct 28 09:52:18.035631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:18.035631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeTestJSON-996475968-project Oct 28 09:52:18.035631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:18.035631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:18.035631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: a4237ca76acd4f7483aaf66e1f5d2eea Oct 28 09:52:18.035631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeTestJSON-996475968-project Oct 28 09:52:18.035631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:18.035631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:18.035631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'c28f1b03-788f-42f6-8577-9c81bffc41ae'} Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:18.037319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:18.040248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:18.042716 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-cbb27c05-8ce8-4843-99a5-c855cd60c486 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Returning 404 to user: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:18.044618 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-cbb27c05-8ce8-4843-99a5-c855cd60c486 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 404 len: 111 microversion: 2.1 time: 0.082208 Oct 28 09:52:18.044618 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 507/919] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:17 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 111 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 09:52:18.061485 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f857497a-8a87-4cf5-a790-0d73048b30bb tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '.version_select at 0x7f271a4890d0>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:18.109611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f857497a-8a87-4cf5-a790-0d73048b30bb tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1965089747" status: 202 len: 0 microversion: 2.1 time: 0.059409 Oct 28 09:52:18.111289 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 508/920] 10.222.0.25 () {58 vars in 1221 bytes} [Fri Oct 28 09:52:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1965089747 => generated 0 bytes in 61 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:52:18.142526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3cea06d3-46e3-4f7b-b8e1-2d9805af10c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:18.158640 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3cea06d3-46e3-4f7b-b8e1-2d9805af10c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:18.192599 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3cea06d3-46e3-4f7b-b8e1-2d9805af10c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:18.195380 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3cea06d3-46e3-4f7b-b8e1-2d9805af10c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:18.198231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3cea06d3-46e3-4f7b-b8e1-2d9805af10c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:18.199283 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3cea06d3-46e3-4f7b-b8e1-2d9805af10c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:18.208088 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3cea06d3-46e3-4f7b-b8e1-2d9805af10c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:18.208704 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3cea06d3-46e3-4f7b-b8e1-2d9805af10c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:18.257449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:18.282556 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3cea06d3-46e3-4f7b-b8e1-2d9805af10c1 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.164500 Oct 28 09:52:18.284785 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 509/921] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:52:18 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/os-volume_attachments => generated 197 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:52:18.499327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0a9c9751-9893-4647-9046-32e2de9fa6cb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:18.500418 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0a9c9751-9893-4647-9046-32e2de9fa6cb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:18.525405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a9c9751-9893-4647-9046-32e2de9fa6cb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:18.527914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a9c9751-9893-4647-9046-32e2de9fa6cb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:18.529200 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a9c9751-9893-4647-9046-32e2de9fa6cb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:18.534195 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a9c9751-9893-4647-9046-32e2de9fa6cb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:18.535615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a9c9751-9893-4647-9046-32e2de9fa6cb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:18.542111 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0a9c9751-9893-4647-9046-32e2de9fa6cb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:19.054858 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0a9c9751-9893-4647-9046-32e2de9fa6cb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:19.073104 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0a9c9751-9893-4647-9046-32e2de9fa6cb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:19.075290 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0a9c9751-9893-4647-9046-32e2de9fa6cb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:19.076879 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0a9c9751-9893-4647-9046-32e2de9fa6cb tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 0.582702 Oct 28 09:52:19.081710 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 510/922] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:18 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:52:19.302071 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3848aa13-546d-4876-9c16-74c278893528 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:19.306064 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3848aa13-546d-4876-9c16-74c278893528 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:19.329477 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3848aa13-546d-4876-9c16-74c278893528 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:19.333868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3848aa13-546d-4876-9c16-74c278893528 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:19.335011 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3848aa13-546d-4876-9c16-74c278893528 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:19.341001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3848aa13-546d-4876-9c16-74c278893528 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:19.341869 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3848aa13-546d-4876-9c16-74c278893528 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:19.344086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3848aa13-546d-4876-9c16-74c278893528 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:19.402322 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3848aa13-546d-4876-9c16-74c278893528 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.106025 Oct 28 09:52:19.403854 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 511/923] 10.222.0.25 () {58 vars in 1302 bytes} [Fri Oct 28 09:52:19 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70/os-volume_attachments => generated 25 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 09:52:19.511858 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2fd9d52c-5873-4476-ad03-2b2e6714ee91 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:19.512772 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2fd9d52c-5873-4476-ad03-2b2e6714ee91 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:19.549597 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2fd9d52c-5873-4476-ad03-2b2e6714ee91 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:19.553059 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2fd9d52c-5873-4476-ad03-2b2e6714ee91 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:19.553454 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2fd9d52c-5873-4476-ad03-2b2e6714ee91 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:19.567741 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2fd9d52c-5873-4476-ad03-2b2e6714ee91 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:19.568670 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2fd9d52c-5873-4476-ad03-2b2e6714ee91 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:19.569315 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2fd9d52c-5873-4476-ad03-2b2e6714ee91 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:19.616848 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2fd9d52c-5873-4476-ad03-2b2e6714ee91 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Going to try to terminate instance {{(pid=127030) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:52:19.763797 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2fd9d52c-5873-4476-ad03-2b2e6714ee91 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 204 len: 0 microversion: 2.1 time: 0.256911 Oct 28 09:52:19.765179 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 512/924] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:52:19 2022] DELETE /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:52:19.774723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b668b1a4-0210-4b2d-9fb0-5b83f0bdbd6c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:19.775448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b668b1a4-0210-4b2d-9fb0-5b83f0bdbd6c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:19.785202 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b668b1a4-0210-4b2d-9fb0-5b83f0bdbd6c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:19.785776 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b668b1a4-0210-4b2d-9fb0-5b83f0bdbd6c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:19.786206 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b668b1a4-0210-4b2d-9fb0-5b83f0bdbd6c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:19.786981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b668b1a4-0210-4b2d-9fb0-5b83f0bdbd6c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:19.787441 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b668b1a4-0210-4b2d-9fb0-5b83f0bdbd6c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:19.787848 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b668b1a4-0210-4b2d-9fb0-5b83f0bdbd6c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:20.652911 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b668b1a4-0210-4b2d-9fb0-5b83f0bdbd6c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:20.664430 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b668b1a4-0210-4b2d-9fb0-5b83f0bdbd6c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1682 microversion: 2.1 time: 0.894394 Oct 28 09:52:20.665255 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 513/925] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:19 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1682 bytes in 896 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:52:20.671621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-90cb74ff-a01b-472f-954c-ea2a759148c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:20.672199 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-90cb74ff-a01b-472f-954c-ea2a759148c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:20.679876 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-90cb74ff-a01b-472f-954c-ea2a759148c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:20.680791 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-90cb74ff-a01b-472f-954c-ea2a759148c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:20.681416 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-90cb74ff-a01b-472f-954c-ea2a759148c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:20.682349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-90cb74ff-a01b-472f-954c-ea2a759148c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:20.682986 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-90cb74ff-a01b-472f-954c-ea2a759148c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:20.683536 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-90cb74ff-a01b-472f-954c-ea2a759148c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:21.238974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-90cb74ff-a01b-472f-954c-ea2a759148c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:21.268449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-90cb74ff-a01b-472f-954c-ea2a759148c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:21.272480 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-90cb74ff-a01b-472f-954c-ea2a759148c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:21.276676 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-90cb74ff-a01b-472f-954c-ea2a759148c7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 0.609729 Oct 28 09:52:21.278659 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 514/926] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:20 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:52:21.727711 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3005ad83-d2f5-4a53-89c7-cacab2b58995 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Action: 'create', calling method: .version_select at 0x7f271dd3cc10>, body: {"keypair": {"name": "tempest-TestShelveInstance-85316997", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHBjwoeZDqGFKwzTduulqQKVB2AWsieGpNDM8qpA+IKap30UUVYFDxlYUDlgaQ0PxL5BF8+gyqS0fMO+B5qV2n/Q0yZm8k5r5F5irpP9tFzUBIgGi5Aoz13InS9e/vMeCA=="}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:52:21.776107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-3005ad83-d2f5-4a53-89c7-cacab2b58995 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Getting quotas for project 937954dbeac043c29259ada2924c4953. Resources: {'key_pairs'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:52:21.781625 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-3005ad83-d2f5-4a53-89c7-cacab2b58995 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Getting quotas for user f14c3f34010b41ef9be2b9fd2aa98cd9 and project 937954dbeac043c29259ada2924c4953. Resources: {'key_pairs'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:52:21.810752 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3005ad83-d2f5-4a53-89c7-cacab2b58995 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 395 microversion: 2.1 time: 0.529134 Oct 28 09:52:21.812410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 515/927] 10.222.0.25 () {60 vars in 1159 bytes} [Fri Oct 28 09:52:21 2022] POST /compute/v2.1/os-keypairs => generated 395 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:52:21.818192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-211fcd96-1ba7-479f-a502-fbe9cecab325 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:21.819479 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-211fcd96-1ba7-479f-a502-fbe9cecab325 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:21.828628 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-211fcd96-1ba7-479f-a502-fbe9cecab325 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:21.829456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-211fcd96-1ba7-479f-a502-fbe9cecab325 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:21.830277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-211fcd96-1ba7-479f-a502-fbe9cecab325 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:21.831323 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-211fcd96-1ba7-479f-a502-fbe9cecab325 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:21.831967 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-211fcd96-1ba7-479f-a502-fbe9cecab325 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:21.832549 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-211fcd96-1ba7-479f-a502-fbe9cecab325 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:22.222893 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-211fcd96-1ba7-479f-a502-fbe9cecab325 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:22.236832 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-211fcd96-1ba7-479f-a502-fbe9cecab325 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1682 microversion: 2.1 time: 0.419523 Oct 28 09:52:22.239712 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 516/928] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:21 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1682 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:52:22.322899 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d85d6c61-da77-4e2c-af45-b92a1bc37af5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:22.323906 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d85d6c61-da77-4e2c-af45-b92a1bc37af5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:22.338978 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d85d6c61-da77-4e2c-af45-b92a1bc37af5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:22.339455 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d85d6c61-da77-4e2c-af45-b92a1bc37af5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:22.341114 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d85d6c61-da77-4e2c-af45-b92a1bc37af5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:22.341907 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d85d6c61-da77-4e2c-af45-b92a1bc37af5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:22.342933 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d85d6c61-da77-4e2c-af45-b92a1bc37af5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:22.343373 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d85d6c61-da77-4e2c-af45-b92a1bc37af5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:22.741034 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d85d6c61-da77-4e2c-af45-b92a1bc37af5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:22.758345 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d85d6c61-da77-4e2c-af45-b92a1bc37af5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:22.759574 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d85d6c61-da77-4e2c-af45-b92a1bc37af5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:22.761563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d85d6c61-da77-4e2c-af45-b92a1bc37af5 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 0.444421 Oct 28 09:52:22.765909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 517/929] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:22 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:52:23.297064 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3180a4c1-bd0f-4476-a4c1-00ff3cc25054 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:23.300467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3180a4c1-bd0f-4476-a4c1-00ff3cc25054 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:23.332284 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3180a4c1-bd0f-4476-a4c1-00ff3cc25054 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:23.335403 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3180a4c1-bd0f-4476-a4c1-00ff3cc25054 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:23.338930 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3180a4c1-bd0f-4476-a4c1-00ff3cc25054 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:23.344669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3180a4c1-bd0f-4476-a4c1-00ff3cc25054 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:23.345658 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3180a4c1-bd0f-4476-a4c1-00ff3cc25054 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:23.346462 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3180a4c1-bd0f-4476-a4c1-00ff3cc25054 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:23.757045 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3180a4c1-bd0f-4476-a4c1-00ff3cc25054 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:23.771008 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3180a4c1-bd0f-4476-a4c1-00ff3cc25054 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1682 microversion: 2.1 time: 0.486688 Oct 28 09:52:23.782429 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 518/930] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:23 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1682 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:52:23.806891 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4bfc53ec-d3e8-44f8-9fa3-5743fa2340c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:23.808201 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4bfc53ec-d3e8-44f8-9fa3-5743fa2340c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:23.815977 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4bfc53ec-d3e8-44f8-9fa3-5743fa2340c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:23.816747 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4bfc53ec-d3e8-44f8-9fa3-5743fa2340c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:23.817375 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4bfc53ec-d3e8-44f8-9fa3-5743fa2340c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:23.818455 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4bfc53ec-d3e8-44f8-9fa3-5743fa2340c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:23.819233 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4bfc53ec-d3e8-44f8-9fa3-5743fa2340c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:23.819931 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4bfc53ec-d3e8-44f8-9fa3-5743fa2340c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:24.495646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-4bfc53ec-d3e8-44f8-9fa3-5743fa2340c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:24.527472 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-4bfc53ec-d3e8-44f8-9fa3-5743fa2340c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:24.534758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-4bfc53ec-d3e8-44f8-9fa3-5743fa2340c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:24.539569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4bfc53ec-d3e8-44f8-9fa3-5743fa2340c1 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 0.736129 Oct 28 09:52:24.541261 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 519/931] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:23 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 740 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:52:24.549617 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ec668fc0-cca2-49d3-b045-4b24d3dcd698 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:24.553138 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ec668fc0-cca2-49d3-b045-4b24d3dcd698 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:24.568652 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ec668fc0-cca2-49d3-b045-4b24d3dcd698 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:24.571789 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ec668fc0-cca2-49d3-b045-4b24d3dcd698 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:24.572286 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ec668fc0-cca2-49d3-b045-4b24d3dcd698 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:24.573122 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ec668fc0-cca2-49d3-b045-4b24d3dcd698 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:24.578569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ec668fc0-cca2-49d3-b045-4b24d3dcd698 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:24.582315 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ec668fc0-cca2-49d3-b045-4b24d3dcd698 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:24.609268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-ec668fc0-cca2-49d3-b045-4b24d3dcd698 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] HTTP exception thrown: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ec668fc0-cca2-49d3-b045-4b24d3dcd698 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Request method failure captured: Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: DELETE /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae HTTP/1.1 Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sr9rY_ITV7HhEadU-8kVwknlsO0ILZxCejbC0tv-IXJnixigZ0yNp9FPJ0cREY_BiaBySs0qEXDy-B2HhLgm_7R6Uo426RjnExTKsP2_sj58xycvGdyeCTUklJQqy6gTOR8ZMSLaSiUs2I6gLtF7HzqTatq6lKFNab2p6V5DQOaCQ Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: f2fcce993e6b4388bde493f0702fd49d Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:24.611753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:24.613229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f2fcce993e6b4388bde493f0702fd49d"}]}] Oct 28 09:52:24.613229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:24.613229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: f2fcce993e6b4388bde493f0702fd49d Oct 28 09:52:24.613229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:24.613229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeTestJSON-996475968-project Oct 28 09:52:24.613229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:24.613229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:24.613229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: a4237ca76acd4f7483aaf66e1f5d2eea Oct 28 09:52:24.613229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeTestJSON-996475968-project Oct 28 09:52:24.613229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:24.613229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:24.613229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'c28f1b03-788f-42f6-8577-9c81bffc41ae'} Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:24.613988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:24.615359 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:24.619140 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ec668fc0-cca2-49d3-b045-4b24d3dcd698 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Returning 404 to user: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:24.620308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ec668fc0-cca2-49d3-b045-4b24d3dcd698 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 404 len: 111 microversion: 2.1 time: 0.077009 Oct 28 09:52:24.621537 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 520/932] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:52:24 2022] DELETE /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:52:24.627731 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-55d868a6-7ec7-4dfb-b501-c5db6d74e643 req-2295bed5-169a-4e61-adbd-4d57600c67ec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9d28bd27-0d1c-45d2-8d9d-81595af44f70", "name": "network-vif-deleted", "tag": "ecc67eeb-ed37-4710-a977-acceff867b07"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:52:24.639712 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-55d868a6-7ec7-4dfb-b501-c5db6d74e643 req-2295bed5-169a-4e61-adbd-4d57600c67ec service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:24.640310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-55d868a6-7ec7-4dfb-b501-c5db6d74e643 req-2295bed5-169a-4e61-adbd-4d57600c67ec service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:24.640853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-55d868a6-7ec7-4dfb-b501-c5db6d74e643 req-2295bed5-169a-4e61-adbd-4d57600c67ec service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:24.664713 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-55d868a6-7ec7-4dfb-b501-c5db6d74e643 req-2295bed5-169a-4e61-adbd-4d57600c67ec service nova] Creating event network-vif-deleted:ecc67eeb-ed37-4710-a977-acceff867b07 for instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 on hv-cinder-zed-3 Oct 28 09:52:24.671930 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-55d868a6-7ec7-4dfb-b501-c5db6d74e643 req-2295bed5-169a-4e61-adbd-4d57600c67ec service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047311 Oct 28 09:52:24.672981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 521/933] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:52:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:52:24.680239 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4e111f2a-3b5a-4783-8494-8ab5ed733dc9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:24.681245 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4e111f2a-3b5a-4783-8494-8ab5ed733dc9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: c28f1b03-788f-42f6-8577-9c81bffc41ae] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:24.696337 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e111f2a-3b5a-4783-8494-8ab5ed733dc9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:24.697240 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e111f2a-3b5a-4783-8494-8ab5ed733dc9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:24.698041 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e111f2a-3b5a-4783-8494-8ab5ed733dc9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:24.699257 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e111f2a-3b5a-4783-8494-8ab5ed733dc9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:24.701209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e111f2a-3b5a-4783-8494-8ab5ed733dc9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:24.702253 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4e111f2a-3b5a-4783-8494-8ab5ed733dc9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:24.728289 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-4e111f2a-3b5a-4783-8494-8ab5ed733dc9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] HTTP exception thrown: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4e111f2a-3b5a-4783-8494-8ab5ed733dc9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Request method failure captured: Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae HTTP/1.1 Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sr9rY_ITV7HhEadU-8kVwknlsO0ILZxCejbC0tv-IXJnixigZ0yNp9FPJ0cREY_BiaBySs0qEXDy-B2HhLgm_7R6Uo426RjnExTKsP2_sj58xycvGdyeCTUklJQqy6gTOR8ZMSLaSiUs2I6gLtF7HzqTatq6lKFNab2p6V5DQOaCQ Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: f2fcce993e6b4388bde493f0702fd49d Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:24.730276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:24.732091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f2fcce993e6b4388bde493f0702fd49d"}]}] Oct 28 09:52:24.732091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:24.732091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: f2fcce993e6b4388bde493f0702fd49d Oct 28 09:52:24.732091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:24.732091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeTestJSON-996475968-project Oct 28 09:52:24.732091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:24.732091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:24.732091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: a4237ca76acd4f7483aaf66e1f5d2eea Oct 28 09:52:24.732091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeTestJSON-996475968-project Oct 28 09:52:24.732091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:24.732091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:24.732091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'c28f1b03-788f-42f6-8577-9c81bffc41ae'} Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:24.733669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. Oct 28 09:52:24.735291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:24.736595 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4e111f2a-3b5a-4783-8494-8ab5ed733dc9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Returning 404 to user: Instance c28f1b03-788f-42f6-8577-9c81bffc41ae could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:24.737583 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4e111f2a-3b5a-4783-8494-8ab5ed733dc9 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae" status: 404 len: 111 microversion: 2.1 time: 0.063088 Oct 28 09:52:24.738863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 522/934] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:24 2022] GET /compute/v2.1/servers/c28f1b03-788f-42f6-8577-9c81bffc41ae => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:52:24.804977 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:24.806054 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:24.828188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:24.837213 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:24.838268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:24.843322 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:24.843896 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:24.849615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:24.898797 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:24.899298 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:24.899743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:24.900205 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lazy-loading 'fault' on Instance uuid 9d28bd27-0d1c-45d2-8d9d-81595af44f70 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:52:24.999026 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65d298cca0f478eba502edf5bd13954', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4f351d016ace4e369be852bb7aa916fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:25.012168 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-41885364-06c6-48c6-822e-f63bd5019b7f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 200 len: 1331 microversion: 2.1 time: 0.212598 Oct 28 09:52:25.013258 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 523/935] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:24 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 1331 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:52:25.574383 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-45896ecb-7c97-4ec9-ad60-ff3a3c6fc744 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:25.580217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-45896ecb-7c97-4ec9-ad60-ff3a3c6fc744 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:25.612917 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-45896ecb-7c97-4ec9-ad60-ff3a3c6fc744 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:25.613249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-45896ecb-7c97-4ec9-ad60-ff3a3c6fc744 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:25.613694 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-45896ecb-7c97-4ec9-ad60-ff3a3c6fc744 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:25.614460 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-45896ecb-7c97-4ec9-ad60-ff3a3c6fc744 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:25.614904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-45896ecb-7c97-4ec9-ad60-ff3a3c6fc744 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:25.622059 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-45896ecb-7c97-4ec9-ad60-ff3a3c6fc744 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:25.974762 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-45896ecb-7c97-4ec9-ad60-ff3a3c6fc744 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:25.999860 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-45896ecb-7c97-4ec9-ad60-ff3a3c6fc744 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:26.008872 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-45896ecb-7c97-4ec9-ad60-ff3a3c6fc744 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:26.010682 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-45896ecb-7c97-4ec9-ad60-ff3a3c6fc744 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 0.444653 Oct 28 09:52:26.011914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 524/936] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:25 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:52:26.021248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-d576af58-4eec-4c7d-870a-625d5443bc99 req-859850dd-59ea-44e5-a89f-fd52f9d20f3b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b68e375d-f7ab-4d35-a8c7-081596d0e6e2", "name": "network-vif-plugged", "status": "completed", "tag": "533de8d8-0438-4dd2-946d-f12312377670"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:52:26.050022 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-d576af58-4eec-4c7d-870a-625d5443bc99 req-859850dd-59ea-44e5-a89f-fd52f9d20f3b service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:26.051369 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-d576af58-4eec-4c7d-870a-625d5443bc99 req-859850dd-59ea-44e5-a89f-fd52f9d20f3b service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:26.052550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-d576af58-4eec-4c7d-870a-625d5443bc99 req-859850dd-59ea-44e5-a89f-fd52f9d20f3b service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:26.073569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-d576af58-4eec-4c7d-870a-625d5443bc99 req-859850dd-59ea-44e5-a89f-fd52f9d20f3b service nova] Creating event network-vif-plugged:533de8d8-0438-4dd2-946d-f12312377670 for instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 on hv-cinder-zed-3 Oct 28 09:52:26.085989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-d576af58-4eec-4c7d-870a-625d5443bc99 req-859850dd-59ea-44e5-a89f-fd52f9d20f3b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071714 Oct 28 09:52:26.087714 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 525/937] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:52:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:52:26.098079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-daee882f-3547-4b5c-b1d7-c4c8473b0009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:26.099469 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-daee882f-3547-4b5c-b1d7-c4c8473b0009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:26.111344 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-daee882f-3547-4b5c-b1d7-c4c8473b0009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:26.112178 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-daee882f-3547-4b5c-b1d7-c4c8473b0009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:26.112930 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-daee882f-3547-4b5c-b1d7-c4c8473b0009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:26.115715 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-daee882f-3547-4b5c-b1d7-c4c8473b0009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:26.116766 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-daee882f-3547-4b5c-b1d7-c4c8473b0009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:26.117741 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-daee882f-3547-4b5c-b1d7-c4c8473b0009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:26.168027 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-daee882f-3547-4b5c-b1d7-c4c8473b0009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-daee882f-3547-4b5c-b1d7-c4c8473b0009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 HTTP/1.1 Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:26.172758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:26.176051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:52:26.176051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:26.176051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:26.176051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:26.176051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:26.176051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:26.176051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:26.176051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:52:26.176051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:26.176051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:26.176051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:26.176051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '9d28bd27-0d1c-45d2-8d9d-81595af44f70'} Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:26.178079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:26.181758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.188024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-daee882f-3547-4b5c-b1d7-c4c8473b0009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 404 to user: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:26.188024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-daee882f-3547-4b5c-b1d7-c4c8473b0009 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 404 len: 111 microversion: 2.1 time: 0.087241 Oct 28 09:52:26.188024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 526/938] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:26 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 111 bytes in 89 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:52:26.233437 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ed770735-0818-4640-9095-c246a63c3416 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:26.234107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ed770735-0818-4640-9095-c246a63c3416 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:26.253451 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ed770735-0818-4640-9095-c246a63c3416 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:26.254586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ed770735-0818-4640-9095-c246a63c3416 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:26.256391 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ed770735-0818-4640-9095-c246a63c3416 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:26.258220 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ed770735-0818-4640-9095-c246a63c3416 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:26.259505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ed770735-0818-4640-9095-c246a63c3416 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:26.260682 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ed770735-0818-4640-9095-c246a63c3416 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:26.308732 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-ed770735-0818-4640-9095-c246a63c3416 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] HTTP exception thrown: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ed770735-0818-4640-9095-c246a63c3416 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Request method failure captured: Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: DELETE /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d HTTP/1.1 Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sr9rY_ITV7HhEadU-8kVwknlsO0ILZxCejbC0tv-IXJnixigZ0yNp9FPJ0cREY_BiaBySs0qEXDy-B2HhLgm_7R6Uo426RjnExTKsP2_sj58xycvGdyeCTUklJQqy6gTOR8ZMSLaSiUs2I6gLtF7HzqTatq6lKFNab2p6V5DQOaCQ Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: f2fcce993e6b4388bde493f0702fd49d Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:26.313217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:26.318427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f2fcce993e6b4388bde493f0702fd49d"}]}] Oct 28 09:52:26.318427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:26.318427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: f2fcce993e6b4388bde493f0702fd49d Oct 28 09:52:26.318427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:26.318427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeTestJSON-996475968-project Oct 28 09:52:26.318427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:26.318427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:26.318427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: a4237ca76acd4f7483aaf66e1f5d2eea Oct 28 09:52:26.318427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeTestJSON-996475968-project Oct 28 09:52:26.318427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:26.318427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:52:26.318427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'b981eca6-4d6f-4e48-bf62-016833f2885d'} Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:26.327890 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:52:26.333971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.337087 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ed770735-0818-4640-9095-c246a63c3416 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Returning 404 to user: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:26.337087 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ed770735-0818-4640-9095-c246a63c3416 tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "DELETE /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 404 len: 111 microversion: 2.1 time: 0.134743 Oct 28 09:52:26.337087 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 527/939] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:52:26 2022] DELETE /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 111 bytes in 143 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:52:26.344645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4f3474c0-a265-4b93-a886-41269e004f6f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '.version_select at 0x7f271a456f70>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:26.378621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4f3474c0-a265-4b93-a886-41269e004f6f tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-530370620" status: 202 len: 0 microversion: 2.1 time: 0.055771 Oct 28 09:52:26.379587 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 528/940] 10.222.0.25 () {58 vars in 1218 bytes} [Fri Oct 28 09:52:26 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-530370620 => generated 0 bytes in 58 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:52:26.397694 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-daec88c8-5901-4754-a837-8aa8e11988af tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:26.403190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-daec88c8-5901-4754-a837-8aa8e11988af tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] [instance: b981eca6-4d6f-4e48-bf62-016833f2885d] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:26.416271 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-daec88c8-5901-4754-a837-8aa8e11988af tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:26.417590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-daec88c8-5901-4754-a837-8aa8e11988af tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:26.418713 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-daec88c8-5901-4754-a837-8aa8e11988af tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:26.420333 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-daec88c8-5901-4754-a837-8aa8e11988af tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:26.424210 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-daec88c8-5901-4754-a837-8aa8e11988af tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:26.425269 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-daec88c8-5901-4754-a837-8aa8e11988af tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:26.459087 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-daec88c8-5901-4754-a837-8aa8e11988af tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] HTTP exception thrown: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-daec88c8-5901-4754-a837-8aa8e11988af tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Request method failure captured: Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d HTTP/1.1 Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sr9rY_ITV7HhEadU-8kVwknlsO0ILZxCejbC0tv-IXJnixigZ0yNp9FPJ0cREY_BiaBySs0qEXDy-B2HhLgm_7R6Uo426RjnExTKsP2_sj58xycvGdyeCTUklJQqy6gTOR8ZMSLaSiUs2I6gLtF7HzqTatq6lKFNab2p6V5DQOaCQ Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: f2fcce993e6b4388bde493f0702fd49d Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:26.464563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:26.468192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_f2fcce993e6b4388bde493f0702fd49d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f2fcce993e6b4388bde493f0702fd49d"}]}] Oct 28 09:52:26.468192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:26.468192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: f2fcce993e6b4388bde493f0702fd49d Oct 28 09:52:26.468192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeTestJSON-996475968 Oct 28 09:52:26.468192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeTestJSON-996475968-project Oct 28 09:52:26.468192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:26.468192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:26.468192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: a4237ca76acd4f7483aaf66e1f5d2eea Oct 28 09:52:26.468192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeTestJSON-996475968-project Oct 28 09:52:26.468192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:26.468192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:52:26.468192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'b981eca6-4d6f-4e48-bf62-016833f2885d'} Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:26.469811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. Oct 28 09:52:26.472693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:26.477534 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-daec88c8-5901-4754-a837-8aa8e11988af tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] Returning 404 to user: Instance b981eca6-4d6f-4e48-bf62-016833f2885d could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:26.479808 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-daec88c8-5901-4754-a837-8aa8e11988af tempest-AttachVolumeTestJSON-996475968 tempest-AttachVolumeTestJSON-996475968-project] 10.222.0.25 "GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d" status: 404 len: 111 microversion: 2.1 time: 0.097437 Oct 28 09:52:26.488437 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 529/941] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:26 2022] GET /compute/v2.1/servers/b981eca6-4d6f-4e48-bf62-016833f2885d => generated 111 bytes in 107 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:52:27.066148 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-91d11095-9b8f-4a01-806e-0684b074ab6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:27.066860 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-91d11095-9b8f-4a01-806e-0684b074ab6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:27.088475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-91d11095-9b8f-4a01-806e-0684b074ab6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:27.089554 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-91d11095-9b8f-4a01-806e-0684b074ab6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:27.090243 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-91d11095-9b8f-4a01-806e-0684b074ab6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:27.091784 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-91d11095-9b8f-4a01-806e-0684b074ab6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:27.093347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-91d11095-9b8f-4a01-806e-0684b074ab6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:27.094670 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-91d11095-9b8f-4a01-806e-0684b074ab6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:27.375438 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-91d11095-9b8f-4a01-806e-0684b074ab6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:27.386239 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:27.393326 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-91d11095-9b8f-4a01-806e-0684b074ab6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:27.395000 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-91d11095-9b8f-4a01-806e-0684b074ab6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:27.396229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-91d11095-9b8f-4a01-806e-0684b074ab6d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 0.334254 Oct 28 09:52:27.398442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 530/942] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:27 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:52:28.431356 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e5d5d67a-417e-457a-9c40-71c73eed5a56 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:28.433347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e5d5d67a-417e-457a-9c40-71c73eed5a56 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:28.452571 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e5d5d67a-417e-457a-9c40-71c73eed5a56 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:28.453693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e5d5d67a-417e-457a-9c40-71c73eed5a56 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:28.454566 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e5d5d67a-417e-457a-9c40-71c73eed5a56 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:28.455756 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e5d5d67a-417e-457a-9c40-71c73eed5a56 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:28.456509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e5d5d67a-417e-457a-9c40-71c73eed5a56 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:28.457595 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e5d5d67a-417e-457a-9c40-71c73eed5a56 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:28.754586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e5d5d67a-417e-457a-9c40-71c73eed5a56 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:28.770600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e5d5d67a-417e-457a-9c40-71c73eed5a56 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:28.773509 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e5d5d67a-417e-457a-9c40-71c73eed5a56 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:28.776439 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e5d5d67a-417e-457a-9c40-71c73eed5a56 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 0.351748 Oct 28 09:52:28.777741 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 531/943] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:28 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:52:29.355665 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-922cc953-8ae7-41e3-985e-f5a87f365756 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:29.356274 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-922cc953-8ae7-41e3-985e-f5a87f365756 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:29.364181 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-922cc953-8ae7-41e3-985e-f5a87f365756 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:29.364938 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-922cc953-8ae7-41e3-985e-f5a87f365756 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:29.365528 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-922cc953-8ae7-41e3-985e-f5a87f365756 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:29.366545 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-922cc953-8ae7-41e3-985e-f5a87f365756 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:29.367225 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-922cc953-8ae7-41e3-985e-f5a87f365756 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:29.367893 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-922cc953-8ae7-41e3-985e-f5a87f365756 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:29.387335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-922cc953-8ae7-41e3-985e-f5a87f365756 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-922cc953-8ae7-41e3-985e-f5a87f365756 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: DELETE /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 HTTP/1.1 Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:29.388816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:29.392275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:52:29.392275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:29.392275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:29.392275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:29.392275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:29.392275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:29.392275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:29.392275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:52:29.392275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:29.392275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:29.392275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:29.392275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '9d28bd27-0d1c-45d2-8d9d-81595af44f70'} Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:29.393972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:29.396619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:29.399242 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-922cc953-8ae7-41e3-985e-f5a87f365756 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 404 to user: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:29.399242 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-922cc953-8ae7-41e3-985e-f5a87f365756 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 404 len: 111 microversion: 2.1 time: 0.038854 Oct 28 09:52:29.399242 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 532/944] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:52:29 2022] DELETE /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 09:52:29.412162 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c930aa75-b75e-4fd8-a622-3e1e2c74caaa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:29.412162 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c930aa75-b75e-4fd8-a622-3e1e2c74caaa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 9d28bd27-0d1c-45d2-8d9d-81595af44f70] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:29.426700 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c930aa75-b75e-4fd8-a622-3e1e2c74caaa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:29.428096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c930aa75-b75e-4fd8-a622-3e1e2c74caaa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:29.429312 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c930aa75-b75e-4fd8-a622-3e1e2c74caaa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:29.431103 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c930aa75-b75e-4fd8-a622-3e1e2c74caaa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:29.432107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c930aa75-b75e-4fd8-a622-3e1e2c74caaa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:29.433177 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c930aa75-b75e-4fd8-a622-3e1e2c74caaa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:29.462086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-c930aa75-b75e-4fd8-a622-3e1e2c74caaa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c930aa75-b75e-4fd8-a622-3e1e2c74caaa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 HTTP/1.1 Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:29.465251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:29.468410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:52:29.468410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:29.468410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:29.468410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:29.468410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:29.468410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:29.468410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:29.468410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:52:29.468410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:29.468410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:29.468410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:29.468410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '9d28bd27-0d1c-45d2-8d9d-81595af44f70'} Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:29.470072 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. Oct 28 09:52:29.474746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:29.482004 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c930aa75-b75e-4fd8-a622-3e1e2c74caaa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 404 to user: Instance 9d28bd27-0d1c-45d2-8d9d-81595af44f70 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:29.482004 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c930aa75-b75e-4fd8-a622-3e1e2c74caaa tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70" status: 404 len: 111 microversion: 2.1 time: 0.067866 Oct 28 09:52:29.482004 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 533/945] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:29 2022] GET /compute/v2.1/servers/9d28bd27-0d1c-45d2-8d9d-81595af44f70 => generated 111 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:52:29.824212 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d33f00be-2fd5-4c15-8f02-923d5489bc9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:29.825170 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d33f00be-2fd5-4c15-8f02-923d5489bc9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:29.833511 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d33f00be-2fd5-4c15-8f02-923d5489bc9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:29.834252 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d33f00be-2fd5-4c15-8f02-923d5489bc9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:29.834824 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d33f00be-2fd5-4c15-8f02-923d5489bc9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:29.835829 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d33f00be-2fd5-4c15-8f02-923d5489bc9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:29.836385 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d33f00be-2fd5-4c15-8f02-923d5489bc9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:29.837132 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d33f00be-2fd5-4c15-8f02-923d5489bc9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:30.064609 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d33f00be-2fd5-4c15-8f02-923d5489bc9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:30.077615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d33f00be-2fd5-4c15-8f02-923d5489bc9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:30.079351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-d33f00be-2fd5-4c15-8f02-923d5489bc9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:30.080648 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d33f00be-2fd5-4c15-8f02-923d5489bc9f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1729 microversion: 2.20 time: 0.261224 Oct 28 09:52:30.081232 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 534/946] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:29 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1729 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 09:52:30.785723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-30498d09-5c86-4b78-ab95-34c60bade93a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:30.786614 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-30498d09-5c86-4b78-ab95-34c60bade93a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:30.801232 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-30498d09-5c86-4b78-ab95-34c60bade93a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:30.802102 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-30498d09-5c86-4b78-ab95-34c60bade93a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:30.802923 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-30498d09-5c86-4b78-ab95-34c60bade93a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:30.804201 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-30498d09-5c86-4b78-ab95-34c60bade93a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:30.805003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-30498d09-5c86-4b78-ab95-34c60bade93a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:30.806546 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-30498d09-5c86-4b78-ab95-34c60bade93a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:30.836897 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-30498d09-5c86-4b78-ab95-34c60bade93a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-30498d09-5c86-4b78-ab95-34c60bade93a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: DELETE /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 HTTP/1.1 Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:30.840610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:30.842033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:52:30.842033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:30.842033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:30.842033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:30.842033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:30.842033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:30.842033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:30.842033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:52:30.842033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:30.842033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:30.842033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:52:30.842033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '576e3364-9c2e-4360-873a-7a9ded31e8e3'} Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:30.843559 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:52:30.844842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.846112 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-30498d09-5c86-4b78-ab95-34c60bade93a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 404 to user: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:30.846112 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-30498d09-5c86-4b78-ab95-34c60bade93a tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 404 len: 111 microversion: 2.1 time: 0.064910 Oct 28 09:52:30.849419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 535/947] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:52:30 2022] DELETE /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 09:52:30.860529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ef18ea49-8472-472d-a028-428d8b2b0391 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:30.861675 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ef18ea49-8472-472d-a028-428d8b2b0391 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 576e3364-9c2e-4360-873a-7a9ded31e8e3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:30.874723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef18ea49-8472-472d-a028-428d8b2b0391 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:30.875773 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef18ea49-8472-472d-a028-428d8b2b0391 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:30.876490 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef18ea49-8472-472d-a028-428d8b2b0391 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:30.877779 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef18ea49-8472-472d-a028-428d8b2b0391 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:30.878643 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef18ea49-8472-472d-a028-428d8b2b0391 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:30.879477 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef18ea49-8472-472d-a028-428d8b2b0391 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:30.902785 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-ef18ea49-8472-472d-a028-428d8b2b0391 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ef18ea49-8472-472d-a028-428d8b2b0391 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 HTTP/1.1 Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:30.904615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:30.906435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:52:30.906435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:30.906435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:30.906435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:30.906435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:30.906435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:30.906435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:30.906435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:52:30.906435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:30.906435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:30.906435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:52:30.906435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '576e3364-9c2e-4360-873a-7a9ded31e8e3'} Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:30.907364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. Oct 28 09:52:30.909218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.910540 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ef18ea49-8472-472d-a028-428d8b2b0391 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 404 to user: Instance 576e3364-9c2e-4360-873a-7a9ded31e8e3 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:30.910540 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ef18ea49-8472-472d-a028-428d8b2b0391 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3" status: 404 len: 111 microversion: 2.1 time: 0.055983 Oct 28 09:52:30.912560 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 536/948] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:30 2022] GET /compute/v2.1/servers/576e3364-9c2e-4360-873a-7a9ded31e8e3 => generated 111 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:52:30.921446 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c69eb484-c9d9-487d-a124-0ebb214d55d3 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:30.922383 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c69eb484-c9d9-487d-a124-0ebb214d55d3 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:30.935262 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c69eb484-c9d9-487d-a124-0ebb214d55d3 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:30.936866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c69eb484-c9d9-487d-a124-0ebb214d55d3 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:30.937632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c69eb484-c9d9-487d-a124-0ebb214d55d3 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:30.938728 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c69eb484-c9d9-487d-a124-0ebb214d55d3 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:30.939478 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c69eb484-c9d9-487d-a124-0ebb214d55d3 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:30.940164 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c69eb484-c9d9-487d-a124-0ebb214d55d3 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:30.959308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-c69eb484-c9d9-487d-a124-0ebb214d55d3 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c69eb484-c9d9-487d-a124-0ebb214d55d3 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: DELETE /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b HTTP/1.1 Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:30.961354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:30.962818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:52:30.962818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:30.962818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:30.962818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:30.962818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:30.962818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:30.962818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:30.962818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:52:30.962818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:30.962818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:30.962818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:52:30.962818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'f8edd976-e011-4194-ad94-1160cc01c63b'} Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:30.963621 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:52:30.964902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:30.967497 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c69eb484-c9d9-487d-a124-0ebb214d55d3 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 404 to user: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:30.967497 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c69eb484-c9d9-487d-a124-0ebb214d55d3 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 404 len: 111 microversion: 2.1 time: 0.046370 Oct 28 09:52:30.967497 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 537/949] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:52:30 2022] DELETE /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 09:52:30.985427 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ab085f49-8dcf-4609-bfdc-3f446701221c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:30.989229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ab085f49-8dcf-4609-bfdc-3f446701221c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: f8edd976-e011-4194-ad94-1160cc01c63b] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:31.001731 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab085f49-8dcf-4609-bfdc-3f446701221c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:31.002836 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab085f49-8dcf-4609-bfdc-3f446701221c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:31.003619 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab085f49-8dcf-4609-bfdc-3f446701221c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:31.005105 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab085f49-8dcf-4609-bfdc-3f446701221c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:31.006050 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab085f49-8dcf-4609-bfdc-3f446701221c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:31.006815 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ab085f49-8dcf-4609-bfdc-3f446701221c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:31.027085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-ab085f49-8dcf-4609-bfdc-3f446701221c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ab085f49-8dcf-4609-bfdc-3f446701221c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b HTTP/1.1 Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:31.028524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:31.031145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:52:31.031145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:31.031145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:31.031145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:31.031145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:31.031145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:31.031145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:31.031145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:52:31.031145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:31.031145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:31.031145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:52:31.031145 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'f8edd976-e011-4194-ad94-1160cc01c63b'} Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:31.032467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. Oct 28 09:52:31.034047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:31.036448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ab085f49-8dcf-4609-bfdc-3f446701221c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 404 to user: Instance f8edd976-e011-4194-ad94-1160cc01c63b could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:31.036448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ab085f49-8dcf-4609-bfdc-3f446701221c tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b" status: 404 len: 111 microversion: 2.1 time: 0.061623 Oct 28 09:52:31.036448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 538/950] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:30 2022] GET /compute/v2.1/servers/f8edd976-e011-4194-ad94-1160cc01c63b => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:52:31.131042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ef173b5a-81b5-43fe-8ef3-7b88cfa96fff tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:31.131974 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ef173b5a-81b5-43fe-8ef3-7b88cfa96fff tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:31.134879 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=96,nova_api:DELETE=2,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:31.145905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef173b5a-81b5-43fe-8ef3-7b88cfa96fff tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:31.148262 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef173b5a-81b5-43fe-8ef3-7b88cfa96fff tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:31.150020 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef173b5a-81b5-43fe-8ef3-7b88cfa96fff tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:31.152377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef173b5a-81b5-43fe-8ef3-7b88cfa96fff tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:31.153722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef173b5a-81b5-43fe-8ef3-7b88cfa96fff tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:31.155085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ef173b5a-81b5-43fe-8ef3-7b88cfa96fff tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:31.349834 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ef173b5a-81b5-43fe-8ef3-7b88cfa96fff tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:31.360757 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ef173b5a-81b5-43fe-8ef3-7b88cfa96fff tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:31.362714 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ef173b5a-81b5-43fe-8ef3-7b88cfa96fff tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:31.364859 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ef173b5a-81b5-43fe-8ef3-7b88cfa96fff tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1716 microversion: 2.20 time: 0.239487 Oct 28 09:52:31.366084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 539/951] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:52:31 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1716 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:52:32.240354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-23b653fc-0031-4e2f-a4db-2bb5c1b54685 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:32.242946 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-23b653fc-0031-4e2f-a4db-2bb5c1b54685 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:32.248779 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=4 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:32.259086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-23b653fc-0031-4e2f-a4db-2bb5c1b54685 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:32.272160 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-23b653fc-0031-4e2f-a4db-2bb5c1b54685 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:32.276447 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-23b653fc-0031-4e2f-a4db-2bb5c1b54685 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:32.278108 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-23b653fc-0031-4e2f-a4db-2bb5c1b54685 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:32.280536 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-23b653fc-0031-4e2f-a4db-2bb5c1b54685 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:32.281617 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-23b653fc-0031-4e2f-a4db-2bb5c1b54685 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:32.325530 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-23b653fc-0031-4e2f-a4db-2bb5c1b54685 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-23b653fc-0031-4e2f-a4db-2bb5c1b54685 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: DELETE /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf HTTP/1.1 Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:32.329611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:32.331211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:52:32.331211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:32.331211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:32.331211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:32.331211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:32.331211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:32.331211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:32.331211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:52:32.331211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:32.331211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:32.331211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:52:32.331211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '82790f9d-85c2-4205-b9ff-290df00c2edf'} Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:32.332853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:52:32.334388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:32.335866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-23b653fc-0031-4e2f-a4db-2bb5c1b54685 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 404 to user: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:32.335866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-23b653fc-0031-4e2f-a4db-2bb5c1b54685 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "DELETE /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 404 len: 111 microversion: 2.1 time: 0.098566 Oct 28 09:52:32.335866 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 540/952] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:52:32 2022] DELETE /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 111 bytes in 102 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:52:32.348199 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ba867757-0e58-4731-9f71-fbff74b43844 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:32.349156 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ba867757-0e58-4731-9f71-fbff74b43844 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] [instance: 82790f9d-85c2-4205-b9ff-290df00c2edf] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:32.359613 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ba867757-0e58-4731-9f71-fbff74b43844 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:32.360161 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ba867757-0e58-4731-9f71-fbff74b43844 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:32.360662 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ba867757-0e58-4731-9f71-fbff74b43844 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:32.361679 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ba867757-0e58-4731-9f71-fbff74b43844 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:32.362355 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ba867757-0e58-4731-9f71-fbff74b43844 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:32.363088 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ba867757-0e58-4731-9f71-fbff74b43844 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:32.390655 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-ba867757-0e58-4731-9f71-fbff74b43844 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] HTTP exception thrown: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ba867757-0e58-4731-9f71-fbff74b43844 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Request method failure captured: Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf HTTP/1.1 Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Sq2VeAhh-1HHe6hQSm8pUMetvUlemWy418XqxE5AO_wDwHEn4aahWosLT3AYMWh3X74s0f7qNbOH1g03I0cabq_mcFfIgIA_2aX-KMZvzr4L-a_rIL9yzJ5RwA4nV2w-np1VSY5SNjZXiC89VAW2kOL__s7oQWnkNY5EqEKATIvw4 Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:32.392301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:32.395398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4f351d016ace4e369be852bb7aa916fe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4f351d016ace4e369be852bb7aa916fe"}]}] Oct 28 09:52:32.395398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:32.395398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 4f351d016ace4e369be852bb7aa916fe Oct 28 09:52:32.395398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1305048312 Oct 28 09:52:32.395398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:32.395398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:32.395398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:32.395398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: e65d298cca0f478eba502edf5bd13954 Oct 28 09:52:32.395398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeNegativeTest-1305048312-project Oct 28 09:52:32.395398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:52:32.395398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:52:32.395398 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '82790f9d-85c2-4205-b9ff-290df00c2edf'} Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:32.397035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. Oct 28 09:52:32.400017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:32.403332 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ba867757-0e58-4731-9f71-fbff74b43844 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] Returning 404 to user: Instance 82790f9d-85c2-4205-b9ff-290df00c2edf could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:32.403332 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ba867757-0e58-4731-9f71-fbff74b43844 tempest-AttachVolumeNegativeTest-1305048312 tempest-AttachVolumeNegativeTest-1305048312-project] 10.222.0.25 "GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf" status: 404 len: 111 microversion: 2.1 time: 0.052218 Oct 28 09:52:32.403332 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 541/953] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:32 2022] GET /compute/v2.1/servers/82790f9d-85c2-4205-b9ff-290df00c2edf => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:52:39.823628 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2f78f65b-30b0-49a3-862f-189058f9640d tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] Action: 'create', calling method: .version_select at 0x7f271a3d0670>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1307870480", "metadata": {"Type": "work"}}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:52:40.317939 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2f78f65b-30b0-49a3-862f-189058f9640d tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 0.629883 Oct 28 09:52:40.319366 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 542/954] 10.222.0.25 () {60 vars in 1156 bytes} [Fri Oct 28 09:52:39 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 633 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 28 09:52:40.344815 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c150f3e9-aa03-4b2b-a672-24207e7815b9 tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] Calling method '.version_select at 0x7f271a49d430>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:40.418605 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c150f3e9-aa03-4b2b-a672-24207e7815b9 tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5" status: 200 len: 353 microversion: 2.1 time: 0.083843 Oct 28 09:52:40.419184 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 543/955] 10.222.0.25 () {58 vars in 1245 bytes} [Fri Oct 28 09:52:40 2022] GET /compute/v2.1/os-volumes/ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 => generated 353 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:52:41.441354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1f584615-ac35-4024-9004-4e0d3a5238fd tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] Calling method '.version_select at 0x7f271a4d0550>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:41.514726 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1f584615-ac35-4024-9004-4e0d3a5238fd tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5" status: 200 len: 354 microversion: 2.1 time: 0.083434 Oct 28 09:52:41.516169 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 544/956] 10.222.0.25 () {58 vars in 1245 bytes} [Fri Oct 28 09:52:41 2022] GET /compute/v2.1/os-volumes/ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 => generated 354 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:52:41.537184 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e0373940-d80b-4949-9d6f-1f8f3aab4e81 tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] Calling method '.version_select at 0x7f271a456280>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:41.613473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e0373940-d80b-4949-9d6f-1f8f3aab4e81 tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5" status: 200 len: 354 microversion: 2.1 time: 0.084260 Oct 28 09:52:41.614235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 545/957] 10.222.0.25 () {58 vars in 1245 bytes} [Fri Oct 28 09:52:41 2022] GET /compute/v2.1/os-volumes/ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 => generated 354 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:52:41.633065 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-142176b3-7286-4e0d-b96e-2e15bbfbf6c9 tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] Calling method '.version_select at 0x7f271a502700>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:41.711495 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-142176b3-7286-4e0d-b96e-2e15bbfbf6c9 tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5" status: 202 len: 0 microversion: 2.1 time: 0.082309 Oct 28 09:52:41.712048 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 546/958] 10.222.0.25 () {58 vars in 1248 bytes} [Fri Oct 28 09:52:41 2022] DELETE /compute/v2.1/os-volumes/ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 => generated 0 bytes in 84 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 09:52:41.725590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2db8acf7-66d0-45ca-a52f-7444ee81d289 tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] Calling method '.version_select at 0x7f271a3f31f0>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:41.791877 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2db8acf7-66d0-45ca-a52f-7444ee81d289 tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5" status: 200 len: 353 microversion: 2.1 time: 0.074776 Oct 28 09:52:41.793105 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 547/959] 10.222.0.25 () {58 vars in 1245 bytes} [Fri Oct 28 09:52:41 2022] GET /compute/v2.1/os-volumes/ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 => generated 353 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:52:42.381041 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=5 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:42.383470 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=34 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:42.389098 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1972129400", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-85316997", "security_groups": [{"name": "tempest-secgroup-smoke-1495480626"}], "networks": [{"uuid": "89f037fa-c0bb-4fca-88c2-3f761674a9f5"}], "block_device_mapping_v2": [{"uuid": "c123c2c4-f8d1-4d1b-b73b-ff013e132971", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:52:42.533737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.network.neutron [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] validate_networks() for [('89f037fa-c0bb-4fca-88c2-3f761674a9f5', None, None, None, None, None)] {{(pid=127030) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:52:43.076940 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Flavor limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:52:43.077392 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Image limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:52:43.077867 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Flavor pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:52:43.078274 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Image pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:52:43.078922 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:52:43.089339 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:43.089997 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:43.090467 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:43.091385 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:43.091849 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:43.092822 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:43.113774 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Getting quotas for project 937954dbeac043c29259ada2924c4953. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:52:43.118977 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Getting quotas for user f14c3f34010b41ef9be2b9fd2aa98cd9 and project 937954dbeac043c29259ada2924c4953. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:52:43.210554 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Going to run 1 instances... {{(pid=127030) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:52:43.213246 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c123c2c4-f8d1-4d1b-b73b-ff013e132971',volume_size=None,volume_type=None)] {{(pid=127030) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:52:43.419085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-39125285-e213-4959-a713-b89e6f1699ab tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.039342 Oct 28 09:52:43.420115 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 548/960] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:52:42 2022] POST /compute/v2.1/servers => generated 406 bytes in 1041 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 28 09:52:43.425451 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e5e2fa17-168b-49a2-a0b9-64fa01ae0d0c tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] Calling method '.version_select at 0x7f271a456280>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:43.475779 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-e5e2fa17-168b-49a2-a0b9-64fa01ae0d0c tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] HTTP exception thrown: Volume ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 could not be found. Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e5e2fa17-168b-49a2-a0b9-64fa01ae0d0c tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] Request method failure captured: Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/os-volumes/ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 HTTP/1.1 Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Zn_vNW40MSQu8wwI1KZFXa0Y3VGSie8CRGVNxdhB9isYstRak-TqeSp4JCXu82iffGLglQ3L_6sbochY2JSnxEljroMHkzhZ1gPYjjxzgPQYwxrnHHYXKcadD_g2iehSlEFxrIO41Cv9CUL8CGpoLJDJ8hU7nvflpKaA-BuHWH0k4 Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: cc0b8c4858ad4ccbb73c54cd7012d924 Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-VolumesGetTestJSON-1659354581 Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:52:43.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:52:43.479163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/cc0b8c4858ad4ccbb73c54cd7012d924"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cc0b8c4858ad4ccbb73c54cd7012d924"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_cc0b8c4858ad4ccbb73c54cd7012d924"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cc0b8c4858ad4ccbb73c54cd7012d924"}]}] Oct 28 09:52:43.479163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-VolumesGetTestJSON-1659354581 Oct 28 09:52:43.479163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: cc0b8c4858ad4ccbb73c54cd7012d924 Oct 28 09:52:43.479163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-VolumesGetTestJSON-1659354581 Oct 28 09:52:43.479163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-VolumesGetTestJSON-1659354581-project Oct 28 09:52:43.479163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:52:43.479163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:52:43.479163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 1ace0b58f73c4279adf024cc677330bd Oct 28 09:52:43.479163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-VolumesGetTestJSON-1659354581-project Oct 28 09:52:43.479163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: Oct 28 09:52:43.479163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Volume ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 could not be found. Oct 28 09:52:43.479163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5'} Oct 28 09:52:43.479163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 could not be found. (HTTP 404) (Request-ID: req-dfaf3a96-78f3-4f98-bd28-41b397b824eb) Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:43.480012 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 09:52:43.481211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 could not be found. Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 28 09:52:43.482475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 could not be found. Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 could not be found. Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e5e2fa17-168b-49a2-a0b9-64fa01ae0d0c tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] Returning 404 to user: Volume ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e5e2fa17-168b-49a2-a0b9-64fa01ae0d0c tempest-VolumesGetTestJSON-1659354581 tempest-VolumesGetTestJSON-1659354581-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5" status: 404 len: 109 microversion: 2.1 time: 0.058001 Oct 28 09:52:43.483678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 549/961] 10.222.0.25 () {58 vars in 1245 bytes} [Fri Oct 28 09:52:43 2022] GET /compute/v2.1/os-volumes/ba0b9c70-4a7b-4f7b-9b14-bb7a0ed465e5 => generated 109 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:52:43.487193 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a38a9ab0-0056-4be3-8d36-020b719cfddc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:43.487827 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a38a9ab0-0056-4be3-8d36-020b719cfddc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:43.548143 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a38a9ab0-0056-4be3-8d36-020b719cfddc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:43.556091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a38a9ab0-0056-4be3-8d36-020b719cfddc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1087 microversion: 2.1 time: 0.074096 Oct 28 09:52:43.558195 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 550/962] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:43 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1087 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:52:44.602930 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b77deb39-7415-4ecd-9eb6-64b396c26023 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:44.603842 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b77deb39-7415-4ecd-9eb6-64b396c26023 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:44.612899 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b77deb39-7415-4ecd-9eb6-64b396c26023 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:44.613571 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b77deb39-7415-4ecd-9eb6-64b396c26023 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:44.614187 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b77deb39-7415-4ecd-9eb6-64b396c26023 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:44.615225 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b77deb39-7415-4ecd-9eb6-64b396c26023 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:44.615881 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b77deb39-7415-4ecd-9eb6-64b396c26023 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:44.616497 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b77deb39-7415-4ecd-9eb6-64b396c26023 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:44.715423 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b77deb39-7415-4ecd-9eb6-64b396c26023 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:44.729869 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b77deb39-7415-4ecd-9eb6-64b396c26023 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1185 microversion: 2.1 time: 0.131578 Oct 28 09:52:44.730947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 551/963] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:44 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1185 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:52:45.762227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-77dc8602-8170-495d-88d5-59ebb7784e23 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:45.763159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-77dc8602-8170-495d-88d5-59ebb7784e23 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:45.787240 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-77dc8602-8170-495d-88d5-59ebb7784e23 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:45.787708 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-77dc8602-8170-495d-88d5-59ebb7784e23 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:45.795862 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-77dc8602-8170-495d-88d5-59ebb7784e23 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:45.797800 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-77dc8602-8170-495d-88d5-59ebb7784e23 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:45.798361 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-77dc8602-8170-495d-88d5-59ebb7784e23 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:45.799144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-77dc8602-8170-495d-88d5-59ebb7784e23 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:46.006124 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-77dc8602-8170-495d-88d5-59ebb7784e23 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:46.030372 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-77dc8602-8170-495d-88d5-59ebb7784e23 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1203 microversion: 2.1 time: 0.273955 Oct 28 09:52:46.044770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 552/964] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:45 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1203 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:52:47.067184 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-50b4b294-0c62-486a-85b2-cc796daffaba tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:47.070140 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-50b4b294-0c62-486a-85b2-cc796daffaba tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:47.088276 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-50b4b294-0c62-486a-85b2-cc796daffaba tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:47.089143 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-50b4b294-0c62-486a-85b2-cc796daffaba tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:47.090310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-50b4b294-0c62-486a-85b2-cc796daffaba tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:47.091239 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-50b4b294-0c62-486a-85b2-cc796daffaba tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:47.092606 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-50b4b294-0c62-486a-85b2-cc796daffaba tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:47.093371 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-50b4b294-0c62-486a-85b2-cc796daffaba tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:47.416080 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-50b4b294-0c62-486a-85b2-cc796daffaba tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:47.460460 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-50b4b294-0c62-486a-85b2-cc796daffaba tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.398597 Oct 28 09:52:47.463695 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 553/965] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:47 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:52:47.476314 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-7ee53db1-af86-4199-9a86-2648a55e82fd req-8bdefb61-c87b-426d-8594-14f18d2e0427 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b96d9706-3980-4e3c-8a22-f18b9a7b4a16", "tag": "3d968631-497a-49af-abf4-297aa62504c8"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:52:47.500013 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-7ee53db1-af86-4199-9a86-2648a55e82fd req-8bdefb61-c87b-426d-8594-14f18d2e0427 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:47.501425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-7ee53db1-af86-4199-9a86-2648a55e82fd req-8bdefb61-c87b-426d-8594-14f18d2e0427 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:47.503312 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-7ee53db1-af86-4199-9a86-2648a55e82fd req-8bdefb61-c87b-426d-8594-14f18d2e0427 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:47.533816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-7ee53db1-af86-4199-9a86-2648a55e82fd req-8bdefb61-c87b-426d-8594-14f18d2e0427 service nova] Creating event network-changed:3d968631-497a-49af-abf4-297aa62504c8 for instance b96d9706-3980-4e3c-8a22-f18b9a7b4a16 on hv-cinder-zed-3 Oct 28 09:52:47.539865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-7ee53db1-af86-4199-9a86-2648a55e82fd req-8bdefb61-c87b-426d-8594-14f18d2e0427 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074139 Oct 28 09:52:47.540817 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 554/966] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:52:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:52:48.489482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1d22203f-1424-4aca-8f44-8731202b60d4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:48.490327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1d22203f-1424-4aca-8f44-8731202b60d4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:48.502440 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1d22203f-1424-4aca-8f44-8731202b60d4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:48.504695 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1d22203f-1424-4aca-8f44-8731202b60d4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:48.505248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1d22203f-1424-4aca-8f44-8731202b60d4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:48.506172 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1d22203f-1424-4aca-8f44-8731202b60d4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:48.506865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1d22203f-1424-4aca-8f44-8731202b60d4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:48.507488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1d22203f-1424-4aca-8f44-8731202b60d4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:48.883096 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1d22203f-1424-4aca-8f44-8731202b60d4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:48.897479 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1d22203f-1424-4aca-8f44-8731202b60d4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.412892 Oct 28 09:52:48.899093 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 555/967] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:48 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:52:49.934875 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-56bd8ed0-3d0d-4495-a73a-5dc334466ee4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:49.937310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-56bd8ed0-3d0d-4495-a73a-5dc334466ee4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:49.949888 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-56bd8ed0-3d0d-4495-a73a-5dc334466ee4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:49.951100 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-56bd8ed0-3d0d-4495-a73a-5dc334466ee4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:49.952461 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-56bd8ed0-3d0d-4495-a73a-5dc334466ee4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:49.953422 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-56bd8ed0-3d0d-4495-a73a-5dc334466ee4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:49.954197 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-56bd8ed0-3d0d-4495-a73a-5dc334466ee4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:49.954833 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-56bd8ed0-3d0d-4495-a73a-5dc334466ee4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:50.174486 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-56bd8ed0-3d0d-4495-a73a-5dc334466ee4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:50.196176 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-56bd8ed0-3d0d-4495-a73a-5dc334466ee4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.267760 Oct 28 09:52:50.197372 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 556/968] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:49 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:52:51.247766 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0bc00d9c-bb24-47e8-a30d-d393eb7d56cf tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:51.249435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0bc00d9c-bb24-47e8-a30d-d393eb7d56cf tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:51.258205 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bc00d9c-bb24-47e8-a30d-d393eb7d56cf tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:51.259049 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bc00d9c-bb24-47e8-a30d-d393eb7d56cf tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:51.259478 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bc00d9c-bb24-47e8-a30d-d393eb7d56cf tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:51.260383 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bc00d9c-bb24-47e8-a30d-d393eb7d56cf tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:51.260999 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bc00d9c-bb24-47e8-a30d-d393eb7d56cf tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:51.261891 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0bc00d9c-bb24-47e8-a30d-d393eb7d56cf tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:51.447421 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0bc00d9c-bb24-47e8-a30d-d393eb7d56cf tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:51.463229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0bc00d9c-bb24-47e8-a30d-d393eb7d56cf tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.226342 Oct 28 09:52:51.464317 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 557/969] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:51 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:52:52.508434 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a83f68cb-a53e-444c-a8f2-1d0a311bb54c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:52.511089 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a83f68cb-a53e-444c-a8f2-1d0a311bb54c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:52.525274 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a83f68cb-a53e-444c-a8f2-1d0a311bb54c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:52.526198 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a83f68cb-a53e-444c-a8f2-1d0a311bb54c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:52.527779 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a83f68cb-a53e-444c-a8f2-1d0a311bb54c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:52.529027 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a83f68cb-a53e-444c-a8f2-1d0a311bb54c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:52.530075 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a83f68cb-a53e-444c-a8f2-1d0a311bb54c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:52.531118 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a83f68cb-a53e-444c-a8f2-1d0a311bb54c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:52.775709 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a83f68cb-a53e-444c-a8f2-1d0a311bb54c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:52.786757 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a83f68cb-a53e-444c-a8f2-1d0a311bb54c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.288493 Oct 28 09:52:52.787590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 558/970] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:52 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:52:53.829167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-a364d23e-f72b-4e97-aa6d-4e2af87562a6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:53.830249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-a364d23e-f72b-4e97-aa6d-4e2af87562a6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:53.833157 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:53.846515 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=8 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:53.866751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a364d23e-f72b-4e97-aa6d-4e2af87562a6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:53.866982 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a364d23e-f72b-4e97-aa6d-4e2af87562a6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:53.867589 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a364d23e-f72b-4e97-aa6d-4e2af87562a6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:53.868984 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a364d23e-f72b-4e97-aa6d-4e2af87562a6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:53.874335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a364d23e-f72b-4e97-aa6d-4e2af87562a6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:53.874335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-a364d23e-f72b-4e97-aa6d-4e2af87562a6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:54.147529 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-a364d23e-f72b-4e97-aa6d-4e2af87562a6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:54.161093 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-a364d23e-f72b-4e97-aa6d-4e2af87562a6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.339432 Oct 28 09:52:54.161686 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 559/971] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:53 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:52:55.193405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7e3f1487-0120-4d3e-b9dd-513a50ae2ea0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:55.194504 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7e3f1487-0120-4d3e-b9dd-513a50ae2ea0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:55.205538 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e3f1487-0120-4d3e-b9dd-513a50ae2ea0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:55.206338 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e3f1487-0120-4d3e-b9dd-513a50ae2ea0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:55.207099 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e3f1487-0120-4d3e-b9dd-513a50ae2ea0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:55.208289 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e3f1487-0120-4d3e-b9dd-513a50ae2ea0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:55.209091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e3f1487-0120-4d3e-b9dd-513a50ae2ea0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:55.209832 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7e3f1487-0120-4d3e-b9dd-513a50ae2ea0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:55.415769 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7e3f1487-0120-4d3e-b9dd-513a50ae2ea0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:55.431403 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7e3f1487-0120-4d3e-b9dd-513a50ae2ea0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.245289 Oct 28 09:52:55.432435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 560/972] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:55 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:52:56.456106 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-202546dc-72f5-42e6-9e95-cc0a486ceec0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:56.456850 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-202546dc-72f5-42e6-9e95-cc0a486ceec0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:56.463965 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-202546dc-72f5-42e6-9e95-cc0a486ceec0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:56.464500 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-202546dc-72f5-42e6-9e95-cc0a486ceec0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:56.465204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-202546dc-72f5-42e6-9e95-cc0a486ceec0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:56.465954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-202546dc-72f5-42e6-9e95-cc0a486ceec0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:56.466485 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-202546dc-72f5-42e6-9e95-cc0a486ceec0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:56.466947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-202546dc-72f5-42e6-9e95-cc0a486ceec0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:56.616862 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-202546dc-72f5-42e6-9e95-cc0a486ceec0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:56.625607 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-202546dc-72f5-42e6-9e95-cc0a486ceec0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.173393 Oct 28 09:52:56.626196 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 561/973] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:56 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:52:57.650085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b2fc4d01-0d1a-4714-8c0b-e35033bd8477 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:57.651392 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b2fc4d01-0d1a-4714-8c0b-e35033bd8477 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:57.659252 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b2fc4d01-0d1a-4714-8c0b-e35033bd8477 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:57.659746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b2fc4d01-0d1a-4714-8c0b-e35033bd8477 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:57.660182 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b2fc4d01-0d1a-4714-8c0b-e35033bd8477 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:57.660912 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b2fc4d01-0d1a-4714-8c0b-e35033bd8477 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:57.661369 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b2fc4d01-0d1a-4714-8c0b-e35033bd8477 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:57.661792 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b2fc4d01-0d1a-4714-8c0b-e35033bd8477 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:57.814277 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b2fc4d01-0d1a-4714-8c0b-e35033bd8477 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:57.822662 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b2fc4d01-0d1a-4714-8c0b-e35033bd8477 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.177337 Oct 28 09:52:57.823422 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 562/974] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:57 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:52:58.847552 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e322fef6-7cef-41ca-9c42-d65d70cde272 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:52:58.848604 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e322fef6-7cef-41ca-9c42-d65d70cde272 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:52:58.856773 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e322fef6-7cef-41ca-9c42-d65d70cde272 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:58.857275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e322fef6-7cef-41ca-9c42-d65d70cde272 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:58.857863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e322fef6-7cef-41ca-9c42-d65d70cde272 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:58.858797 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e322fef6-7cef-41ca-9c42-d65d70cde272 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:52:58.859401 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e322fef6-7cef-41ca-9c42-d65d70cde272 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:52:58.870969 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e322fef6-7cef-41ca-9c42-d65d70cde272 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:52:59.031386 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e322fef6-7cef-41ca-9c42-d65d70cde272 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:52:59.039796 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e322fef6-7cef-41ca-9c42-d65d70cde272 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.197913 Oct 28 09:52:59.040390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 563/975] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:52:58 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:00.065487 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-bafc9964-563a-4144-9e92-763b3d5ab5b5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:00.066202 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-bafc9964-563a-4144-9e92-763b3d5ab5b5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:00.073026 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bafc9964-563a-4144-9e92-763b3d5ab5b5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:00.073526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bafc9964-563a-4144-9e92-763b3d5ab5b5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:00.073967 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bafc9964-563a-4144-9e92-763b3d5ab5b5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:00.074734 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bafc9964-563a-4144-9e92-763b3d5ab5b5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:00.075191 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bafc9964-563a-4144-9e92-763b3d5ab5b5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:00.075617 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bafc9964-563a-4144-9e92-763b3d5ab5b5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:00.262513 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-bafc9964-563a-4144-9e92-763b3d5ab5b5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:00.272351 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-bafc9964-563a-4144-9e92-763b3d5ab5b5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.212549 Oct 28 09:53:00.272947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 564/976] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:00 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:01.306307 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-eb45e67f-1e80-46c5-ae94-0978d34dd533 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:01.307394 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-eb45e67f-1e80-46c5-ae94-0978d34dd533 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:01.318086 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eb45e67f-1e80-46c5-ae94-0978d34dd533 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:01.318836 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eb45e67f-1e80-46c5-ae94-0978d34dd533 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:01.319310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eb45e67f-1e80-46c5-ae94-0978d34dd533 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:01.320197 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eb45e67f-1e80-46c5-ae94-0978d34dd533 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:01.320791 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eb45e67f-1e80-46c5-ae94-0978d34dd533 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:01.321285 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-eb45e67f-1e80-46c5-ae94-0978d34dd533 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:01.529833 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-eb45e67f-1e80-46c5-ae94-0978d34dd533 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:01.544751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-eb45e67f-1e80-46c5-ae94-0978d34dd533 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.245645 Oct 28 09:53:01.545803 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 565/977] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:01 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:02.584064 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-13da8844-a11c-4290-a256-04ee5e657af1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:02.586517 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-13da8844-a11c-4290-a256-04ee5e657af1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:02.593897 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-13da8844-a11c-4290-a256-04ee5e657af1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:02.594602 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-13da8844-a11c-4290-a256-04ee5e657af1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:02.595314 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-13da8844-a11c-4290-a256-04ee5e657af1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:02.596199 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-13da8844-a11c-4290-a256-04ee5e657af1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:02.597180 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-13da8844-a11c-4290-a256-04ee5e657af1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:02.597715 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-13da8844-a11c-4290-a256-04ee5e657af1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:02.849755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-13da8844-a11c-4290-a256-04ee5e657af1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:02.872663 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-13da8844-a11c-4290-a256-04ee5e657af1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.290257 Oct 28 09:53:02.874915 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 566/978] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:02 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:53:02.895296 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-bbdef86c-fcd5-4c1f-b81b-4bfd777a0102 req-ceb36e4f-e507-4148-a937-b06c3dd9def6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b96d9706-3980-4e3c-8a22-f18b9a7b4a16", "name": "network-vif-plugged", "status": "completed", "tag": "3d968631-497a-49af-abf4-297aa62504c8"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:53:02.911913 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-bbdef86c-fcd5-4c1f-b81b-4bfd777a0102 req-ceb36e4f-e507-4148-a937-b06c3dd9def6 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:02.913156 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-bbdef86c-fcd5-4c1f-b81b-4bfd777a0102 req-ceb36e4f-e507-4148-a937-b06c3dd9def6 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:02.914181 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-bbdef86c-fcd5-4c1f-b81b-4bfd777a0102 req-ceb36e4f-e507-4148-a937-b06c3dd9def6 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:02.938815 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-bbdef86c-fcd5-4c1f-b81b-4bfd777a0102 req-ceb36e4f-e507-4148-a937-b06c3dd9def6 service nova] Creating event network-vif-plugged:3d968631-497a-49af-abf4-297aa62504c8 for instance b96d9706-3980-4e3c-8a22-f18b9a7b4a16 on hv-cinder-zed-3 Oct 28 09:53:02.948912 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-bbdef86c-fcd5-4c1f-b81b-4bfd777a0102 req-ceb36e4f-e507-4148-a937-b06c3dd9def6 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071608 Oct 28 09:53:02.950482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 567/979] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:53:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:53:03.900734 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-85bb354c-5e53-4ec6-8275-1c0385fcff9a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:03.901730 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-85bb354c-5e53-4ec6-8275-1c0385fcff9a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:03.913432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-85bb354c-5e53-4ec6-8275-1c0385fcff9a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:03.914252 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-85bb354c-5e53-4ec6-8275-1c0385fcff9a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:03.914948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-85bb354c-5e53-4ec6-8275-1c0385fcff9a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:03.916147 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-85bb354c-5e53-4ec6-8275-1c0385fcff9a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:03.916821 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-85bb354c-5e53-4ec6-8275-1c0385fcff9a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:03.917562 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-85bb354c-5e53-4ec6-8275-1c0385fcff9a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:03.947460 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=100 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:53:04.124967 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-85bb354c-5e53-4ec6-8275-1c0385fcff9a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:04.136073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-85bb354c-5e53-4ec6-8275-1c0385fcff9a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1259 microversion: 2.1 time: 0.241225 Oct 28 09:53:04.136838 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 568/980] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:03 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1259 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:05.166611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2ebc9526-ae8d-4755-92aa-89fc10eead38 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:05.169202 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2ebc9526-ae8d-4755-92aa-89fc10eead38 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:05.182554 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ebc9526-ae8d-4755-92aa-89fc10eead38 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:05.183935 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ebc9526-ae8d-4755-92aa-89fc10eead38 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:05.185117 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ebc9526-ae8d-4755-92aa-89fc10eead38 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:05.187485 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ebc9526-ae8d-4755-92aa-89fc10eead38 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:05.188761 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ebc9526-ae8d-4755-92aa-89fc10eead38 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:05.189526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2ebc9526-ae8d-4755-92aa-89fc10eead38 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:05.448622 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2ebc9526-ae8d-4755-92aa-89fc10eead38 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:05.466248 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2ebc9526-ae8d-4755-92aa-89fc10eead38 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1444 microversion: 2.1 time: 0.307487 Oct 28 09:53:05.478442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 569/981] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:05 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1444 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:05.511425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-767195f8-61c8-4b82-a400-0a7d95825f0b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:05.512524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-767195f8-61c8-4b82-a400-0a7d95825f0b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:05.524004 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-767195f8-61c8-4b82-a400-0a7d95825f0b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:05.525235 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-767195f8-61c8-4b82-a400-0a7d95825f0b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:05.526021 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-767195f8-61c8-4b82-a400-0a7d95825f0b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:05.527364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-767195f8-61c8-4b82-a400-0a7d95825f0b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:05.528206 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-767195f8-61c8-4b82-a400-0a7d95825f0b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:05.529020 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-767195f8-61c8-4b82-a400-0a7d95825f0b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:05.772457 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-767195f8-61c8-4b82-a400-0a7d95825f0b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:05.783251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-767195f8-61c8-4b82-a400-0a7d95825f0b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1444 microversion: 2.1 time: 0.278972 Oct 28 09:53:05.784595 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 570/982] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:05 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1444 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:53:09.116171 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-433f3ca7-910b-4d27-bdb9-bf0b86275d02 req-cb680cdc-cb8f-4a11-9242-a73e4f86df82 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b96d9706-3980-4e3c-8a22-f18b9a7b4a16", "tag": "3d968631-497a-49af-abf4-297aa62504c8"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:53:09.134044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-433f3ca7-910b-4d27-bdb9-bf0b86275d02 req-cb680cdc-cb8f-4a11-9242-a73e4f86df82 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:09.134779 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-433f3ca7-910b-4d27-bdb9-bf0b86275d02 req-cb680cdc-cb8f-4a11-9242-a73e4f86df82 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:09.135341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-433f3ca7-910b-4d27-bdb9-bf0b86275d02 req-cb680cdc-cb8f-4a11-9242-a73e4f86df82 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:09.176031 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-433f3ca7-910b-4d27-bdb9-bf0b86275d02 req-cb680cdc-cb8f-4a11-9242-a73e4f86df82 service nova] Creating event network-changed:3d968631-497a-49af-abf4-297aa62504c8 for instance b96d9706-3980-4e3c-8a22-f18b9a7b4a16 on hv-cinder-zed-3 Oct 28 09:53:09.191706 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-433f3ca7-910b-4d27-bdb9-bf0b86275d02 req-cb680cdc-cb8f-4a11-9242-a73e4f86df82 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.079322 Oct 28 09:53:09.193830 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 571/983] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:53:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:53:13.555754 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-11062603-bfa1-46a3-bb36-573d63f3023a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:13.558271 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-11062603-bfa1-46a3-bb36-573d63f3023a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:13.566571 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=62,nova_api:INSERT=3 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:53:13.569396 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11062603-bfa1-46a3-bb36-573d63f3023a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:13.570077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11062603-bfa1-46a3-bb36-573d63f3023a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:13.570812 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11062603-bfa1-46a3-bb36-573d63f3023a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:13.571849 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11062603-bfa1-46a3-bb36-573d63f3023a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:13.572315 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11062603-bfa1-46a3-bb36-573d63f3023a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:13.572801 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11062603-bfa1-46a3-bb36-573d63f3023a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:13.620804 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-11062603-bfa1-46a3-bb36-573d63f3023a tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.076747 Oct 28 09:53:13.621409 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 572/984] 10.222.0.25 () {60 vars in 1343 bytes} [Fri Oct 28 09:53:13 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2/os-volume_attachments => generated 25 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:13.763104 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-11e6bf60-e726-4fe7-907b-893aef32a739 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:13.764860 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-11e6bf60-e726-4fe7-907b-893aef32a739 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:13.771550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11e6bf60-e726-4fe7-907b-893aef32a739 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:13.772240 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11e6bf60-e726-4fe7-907b-893aef32a739 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:13.772790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11e6bf60-e726-4fe7-907b-893aef32a739 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:13.773675 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11e6bf60-e726-4fe7-907b-893aef32a739 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:13.774488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11e6bf60-e726-4fe7-907b-893aef32a739 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:13.775191 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11e6bf60-e726-4fe7-907b-893aef32a739 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:13.816492 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-11e6bf60-e726-4fe7-907b-893aef32a739 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Going to try to terminate instance {{(pid=127030) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:53:13.928433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-11e6bf60-e726-4fe7-907b-893aef32a739 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "DELETE /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 204 len: 0 microversion: 2.20 time: 0.168667 Oct 28 09:53:13.929557 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 573/985] 10.222.0.25 () {60 vars in 1280 bytes} [Fri Oct 28 09:53:13 2022] DELETE /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 28 09:53:13.950821 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f15a2079-3c60-4ba7-b64e-5249f76cefb9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:13.952579 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f15a2079-3c60-4ba7-b64e-5249f76cefb9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:13.967840 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f15a2079-3c60-4ba7-b64e-5249f76cefb9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:13.968696 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f15a2079-3c60-4ba7-b64e-5249f76cefb9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:13.969766 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f15a2079-3c60-4ba7-b64e-5249f76cefb9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:13.971132 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f15a2079-3c60-4ba7-b64e-5249f76cefb9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:13.971987 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f15a2079-3c60-4ba7-b64e-5249f76cefb9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:13.973127 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f15a2079-3c60-4ba7-b64e-5249f76cefb9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:14.185770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f15a2079-3c60-4ba7-b64e-5249f76cefb9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:14.198840 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f15a2079-3c60-4ba7-b64e-5249f76cefb9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:14.201636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f15a2079-3c60-4ba7-b64e-5249f76cefb9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:14.203076 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f15a2079-3c60-4ba7-b64e-5249f76cefb9 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.265404 Oct 28 09:53:14.204171 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 574/986] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:53:13 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:53:15.231601 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6f879d17-742b-4222-a7e5-8d9a098cada6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:15.233183 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6f879d17-742b-4222-a7e5-8d9a098cada6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:15.243228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f879d17-742b-4222-a7e5-8d9a098cada6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:15.243987 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f879d17-742b-4222-a7e5-8d9a098cada6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:15.244605 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f879d17-742b-4222-a7e5-8d9a098cada6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:15.245476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f879d17-742b-4222-a7e5-8d9a098cada6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:15.246102 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f879d17-742b-4222-a7e5-8d9a098cada6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:15.246715 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6f879d17-742b-4222-a7e5-8d9a098cada6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:15.438770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6f879d17-742b-4222-a7e5-8d9a098cada6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:15.454584 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6f879d17-742b-4222-a7e5-8d9a098cada6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:15.458887 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6f879d17-742b-4222-a7e5-8d9a098cada6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:15.461886 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6f879d17-742b-4222-a7e5-8d9a098cada6 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1722 microversion: 2.20 time: 0.234910 Oct 28 09:53:15.463505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 575/987] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:53:15 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1722 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:53:16.503970 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-69fbe253-0a49-4ddf-b2a8-8f67944abc47 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:16.505783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-69fbe253-0a49-4ddf-b2a8-8f67944abc47 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:16.522919 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69fbe253-0a49-4ddf-b2a8-8f67944abc47 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:16.525837 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69fbe253-0a49-4ddf-b2a8-8f67944abc47 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:16.530850 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69fbe253-0a49-4ddf-b2a8-8f67944abc47 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:16.532796 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69fbe253-0a49-4ddf-b2a8-8f67944abc47 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:16.534038 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69fbe253-0a49-4ddf-b2a8-8f67944abc47 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:16.535192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69fbe253-0a49-4ddf-b2a8-8f67944abc47 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:16.668780 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-69fbe253-0a49-4ddf-b2a8-8f67944abc47 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:16.680415 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-69fbe253-0a49-4ddf-b2a8-8f67944abc47 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:16.683048 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-69fbe253-0a49-4ddf-b2a8-8f67944abc47 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:16.684432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-69fbe253-0a49-4ddf-b2a8-8f67944abc47 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1656 microversion: 2.20 time: 0.188792 Oct 28 09:53:16.685755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 576/988] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:53:16 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1656 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:53:17.049386 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-6108ebc1-e75d-48e2-b15e-a15797b79798 req-ea5b2a02-4159-41ed-947a-fbc37a866a11 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b68e375d-f7ab-4d35-a8c7-081596d0e6e2", "name": "network-vif-deleted", "tag": "533de8d8-0438-4dd2-946d-f12312377670"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:53:17.093864 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-6108ebc1-e75d-48e2-b15e-a15797b79798 req-ea5b2a02-4159-41ed-947a-fbc37a866a11 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:17.094997 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-6108ebc1-e75d-48e2-b15e-a15797b79798 req-ea5b2a02-4159-41ed-947a-fbc37a866a11 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:17.098341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-6108ebc1-e75d-48e2-b15e-a15797b79798 req-ea5b2a02-4159-41ed-947a-fbc37a866a11 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:17.137530 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-6108ebc1-e75d-48e2-b15e-a15797b79798 req-ea5b2a02-4159-41ed-947a-fbc37a866a11 service nova] Creating event network-vif-deleted:533de8d8-0438-4dd2-946d-f12312377670 for instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 on hv-cinder-zed-3 Oct 28 09:53:17.148983 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-6108ebc1-e75d-48e2-b15e-a15797b79798 req-ea5b2a02-4159-41ed-947a-fbc37a866a11 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.108520 Oct 28 09:53:17.151338 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 577/989] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:53:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:53:17.717892 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:17.718989 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:17.731985 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:17.732980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:17.733983 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:17.735217 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:17.735998 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:17.736704 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:17.790001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:17.790871 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:17.791423 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:17.791996 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lazy-loading 'fault' on Instance uuid b68e375d-f7ab-4d35-a8c7-081596d0e6e2 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:53:17.847992 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:17.859763 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:17.862851 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '715a782e5a56431e88e4fe5a3b0a62aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29c47b13a2044939b5446ffe5397633a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:17.865183 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-36ad86f9-6bf7-48b7-9281-a088149a662f tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 200 len: 1371 microversion: 2.20 time: 0.154894 Oct 28 09:53:17.870632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 578/990] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:53:17 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 1371 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 09:53:18.892922 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-686be77b-02b5-4a2e-a677-fd903d68b46d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:18.893727 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-686be77b-02b5-4a2e-a677-fd903d68b46d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:18.902705 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-686be77b-02b5-4a2e-a677-fd903d68b46d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:18.903589 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-686be77b-02b5-4a2e-a677-fd903d68b46d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:18.904413 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-686be77b-02b5-4a2e-a677-fd903d68b46d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:18.905670 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-686be77b-02b5-4a2e-a677-fd903d68b46d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:18.906524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-686be77b-02b5-4a2e-a677-fd903d68b46d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:18.907292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-686be77b-02b5-4a2e-a677-fd903d68b46d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:18.934724 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-686be77b-02b5-4a2e-a677-fd903d68b46d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] HTTP exception thrown: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-686be77b-02b5-4a2e-a677-fd903d68b46d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Request method failure captured: Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 HTTP/1.1 Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Skf5aWDdC40CR74ytCMLUExvGa9v-WnWSZgir8FnsqOIPgcQbm8rt7LfqPr3K-OpviTVPDVrFgjoVWp5XTqu5zvyFdSr2QLXhFcNRteklZzYEfWKkCO2Kw-We7GIm7OBNlI_jJkjuNbTKEij2gaBf7wIZ1vIvR2dikCsL6NQ3ueTQ Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Openstack-Nova-Api-Version: 2.20 Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 29c47b13a2044939b5446ffe5397633a Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:53:18.938550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:53:18.940431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/29c47b13a2044939b5446ffe5397633a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/29c47b13a2044939b5446ffe5397633a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_29c47b13a2044939b5446ffe5397633a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/29c47b13a2044939b5446ffe5397633a"}]}] Oct 28 09:53:18.940431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:18.940431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 29c47b13a2044939b5446ffe5397633a Oct 28 09:53:18.940431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:18.940431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeShelveTestJSON-387501597-project Oct 28 09:53:18.940431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:53:18.940431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:53:18.940431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 715a782e5a56431e88e4fe5a3b0a62aa Oct 28 09:53:18.940431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeShelveTestJSON-387501597-project Oct 28 09:53:18.940431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:53:18.940431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:18.940431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'b68e375d-f7ab-4d35-a8c7-081596d0e6e2'} Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:53:18.941349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:18.943425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:18.945946 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-686be77b-02b5-4a2e-a677-fd903d68b46d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Returning 404 to user: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:53:18.945946 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-686be77b-02b5-4a2e-a677-fd903d68b46d tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 404 len: 111 microversion: 2.20 time: 0.053134 Oct 28 09:53:18.945946 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 579/991] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:53:18 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 28 09:53:18.956556 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e2e6150c-1531-4e5c-8df0-e5bbd8173fe7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '.version_select at 0x7f271a565430>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:18.987925 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e2e6150c-1531-4e5c-8df0-e5bbd8173fe7 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-853652994" status: 204 len: 0 microversion: 2.20 time: 0.038412 Oct 28 09:53:18.988977 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 580/992] 10.222.0.25 () {60 vars in 1259 bytes} [Fri Oct 28 09:53:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-853652994 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 28 09:53:21.064134 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e1fa3c18-0a02-4a85-8281-83dcee91f447 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:21.065189 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e1fa3c18-0a02-4a85-8281-83dcee91f447 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:21.072476 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e1fa3c18-0a02-4a85-8281-83dcee91f447 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:21.073036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e1fa3c18-0a02-4a85-8281-83dcee91f447 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:21.073471 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e1fa3c18-0a02-4a85-8281-83dcee91f447 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:21.074190 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e1fa3c18-0a02-4a85-8281-83dcee91f447 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:21.074673 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e1fa3c18-0a02-4a85-8281-83dcee91f447 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:21.075094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e1fa3c18-0a02-4a85-8281-83dcee91f447 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:21.093923 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-e1fa3c18-0a02-4a85-8281-83dcee91f447 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] HTTP exception thrown: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e1fa3c18-0a02-4a85-8281-83dcee91f447 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Request method failure captured: Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: DELETE /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 HTTP/1.1 Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Skf5aWDdC40CR74ytCMLUExvGa9v-WnWSZgir8FnsqOIPgcQbm8rt7LfqPr3K-OpviTVPDVrFgjoVWp5XTqu5zvyFdSr2QLXhFcNRteklZzYEfWKkCO2Kw-We7GIm7OBNlI_jJkjuNbTKEij2gaBf7wIZ1vIvR2dikCsL6NQ3ueTQ Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Openstack-Nova-Api-Version: 2.20 Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 29c47b13a2044939b5446ffe5397633a Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:53:21.096924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:53:21.098488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/29c47b13a2044939b5446ffe5397633a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/29c47b13a2044939b5446ffe5397633a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_29c47b13a2044939b5446ffe5397633a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/29c47b13a2044939b5446ffe5397633a"}]}] Oct 28 09:53:21.098488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:21.098488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 29c47b13a2044939b5446ffe5397633a Oct 28 09:53:21.098488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:21.098488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeShelveTestJSON-387501597-project Oct 28 09:53:21.098488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:53:21.098488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:53:21.098488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 715a782e5a56431e88e4fe5a3b0a62aa Oct 28 09:53:21.098488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeShelveTestJSON-387501597-project Oct 28 09:53:21.098488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:53:21.098488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:21.098488 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'b68e375d-f7ab-4d35-a8c7-081596d0e6e2'} Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:53:21.099680 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:21.101433 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:21.102970 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e1fa3c18-0a02-4a85-8281-83dcee91f447 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Returning 404 to user: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:53:21.102970 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e1fa3c18-0a02-4a85-8281-83dcee91f447 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "DELETE /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 404 len: 111 microversion: 2.20 time: 0.040565 Oct 28 09:53:21.103933 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 581/993] 10.222.0.25 () {60 vars in 1280 bytes} [Fri Oct 28 09:53:21 2022] DELETE /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 28 09:53:21.117702 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-47e991d4-89c1-4f84-97a7-e0cdb6f6fe42 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:21.119085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-47e991d4-89c1-4f84-97a7-e0cdb6f6fe42 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: b68e375d-f7ab-4d35-a8c7-081596d0e6e2] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:21.130463 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-47e991d4-89c1-4f84-97a7-e0cdb6f6fe42 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:21.131361 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-47e991d4-89c1-4f84-97a7-e0cdb6f6fe42 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:21.132146 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-47e991d4-89c1-4f84-97a7-e0cdb6f6fe42 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:21.133403 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-47e991d4-89c1-4f84-97a7-e0cdb6f6fe42 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:21.134258 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-47e991d4-89c1-4f84-97a7-e0cdb6f6fe42 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:21.135100 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-47e991d4-89c1-4f84-97a7-e0cdb6f6fe42 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:21.154678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-47e991d4-89c1-4f84-97a7-e0cdb6f6fe42 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] HTTP exception thrown: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-47e991d4-89c1-4f84-97a7-e0cdb6f6fe42 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Request method failure captured: Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 HTTP/1.1 Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Skf5aWDdC40CR74ytCMLUExvGa9v-WnWSZgir8FnsqOIPgcQbm8rt7LfqPr3K-OpviTVPDVrFgjoVWp5XTqu5zvyFdSr2QLXhFcNRteklZzYEfWKkCO2Kw-We7GIm7OBNlI_jJkjuNbTKEij2gaBf7wIZ1vIvR2dikCsL6NQ3ueTQ Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Openstack-Nova-Api-Version: 2.20 Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 29c47b13a2044939b5446ffe5397633a Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:53:21.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:53:21.158719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/29c47b13a2044939b5446ffe5397633a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/29c47b13a2044939b5446ffe5397633a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_29c47b13a2044939b5446ffe5397633a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/29c47b13a2044939b5446ffe5397633a"}]}] Oct 28 09:53:21.158719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:21.158719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 29c47b13a2044939b5446ffe5397633a Oct 28 09:53:21.158719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:21.158719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeShelveTestJSON-387501597-project Oct 28 09:53:21.158719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:53:21.158719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:53:21.158719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 715a782e5a56431e88e4fe5a3b0a62aa Oct 28 09:53:21.158719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeShelveTestJSON-387501597-project Oct 28 09:53:21.158719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:53:21.158719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:21.158719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'b68e375d-f7ab-4d35-a8c7-081596d0e6e2'} Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:53:21.160327 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. Oct 28 09:53:21.163868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:21.167978 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-47e991d4-89c1-4f84-97a7-e0cdb6f6fe42 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Returning 404 to user: Instance b68e375d-f7ab-4d35-a8c7-081596d0e6e2 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:53:21.167978 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-47e991d4-89c1-4f84-97a7-e0cdb6f6fe42 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2" status: 404 len: 111 microversion: 2.20 time: 0.048192 Oct 28 09:53:21.167978 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 582/994] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:53:21 2022] GET /compute/v2.1/servers/b68e375d-f7ab-4d35-a8c7-081596d0e6e2 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 28 09:53:22.395738 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b1b123ec-ddb8-48d7-b08c-6616408c5d01 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:22.396776 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b1b123ec-ddb8-48d7-b08c-6616408c5d01 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:22.408419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b1b123ec-ddb8-48d7-b08c-6616408c5d01 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:22.411040 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b1b123ec-ddb8-48d7-b08c-6616408c5d01 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:22.411604 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b1b123ec-ddb8-48d7-b08c-6616408c5d01 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:22.412466 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b1b123ec-ddb8-48d7-b08c-6616408c5d01 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:22.413035 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b1b123ec-ddb8-48d7-b08c-6616408c5d01 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:22.414020 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b1b123ec-ddb8-48d7-b08c-6616408c5d01 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:22.433100 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-b1b123ec-ddb8-48d7-b08c-6616408c5d01 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] HTTP exception thrown: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b1b123ec-ddb8-48d7-b08c-6616408c5d01 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Request method failure captured: Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: DELETE /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e HTTP/1.1 Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Skf5aWDdC40CR74ytCMLUExvGa9v-WnWSZgir8FnsqOIPgcQbm8rt7LfqPr3K-OpviTVPDVrFgjoVWp5XTqu5zvyFdSr2QLXhFcNRteklZzYEfWKkCO2Kw-We7GIm7OBNlI_jJkjuNbTKEij2gaBf7wIZ1vIvR2dikCsL6NQ3ueTQ Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Openstack-Nova-Api-Version: 2.20 Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 29c47b13a2044939b5446ffe5397633a Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:53:22.448110 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:53:22.451790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/29c47b13a2044939b5446ffe5397633a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/29c47b13a2044939b5446ffe5397633a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_29c47b13a2044939b5446ffe5397633a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/29c47b13a2044939b5446ffe5397633a"}]}] Oct 28 09:53:22.451790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:22.451790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 29c47b13a2044939b5446ffe5397633a Oct 28 09:53:22.451790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:22.451790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeShelveTestJSON-387501597-project Oct 28 09:53:22.451790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:53:22.451790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:53:22.451790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 715a782e5a56431e88e4fe5a3b0a62aa Oct 28 09:53:22.451790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeShelveTestJSON-387501597-project Oct 28 09:53:22.451790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:53:22.451790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:53:22.451790 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'c716d5fd-f257-4f6a-8f52-18ad59e5c31e'} Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:53:22.453745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:53:22.458568 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:22.464159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b1b123ec-ddb8-48d7-b08c-6616408c5d01 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Returning 404 to user: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:53:22.464159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b1b123ec-ddb8-48d7-b08c-6616408c5d01 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 404 len: 111 microversion: 2.20 time: 0.058546 Oct 28 09:53:22.464159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 583/995] 10.222.0.25 () {60 vars in 1280 bytes} [Fri Oct 28 09:53:22 2022] DELETE /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 28 09:53:22.464159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-abbe81bf-7998-42f5-ba4a-5703b42b3f88 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:22.464159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-abbe81bf-7998-42f5-ba4a-5703b42b3f88 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] [instance: c716d5fd-f257-4f6a-8f52-18ad59e5c31e] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:22.469801 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-abbe81bf-7998-42f5-ba4a-5703b42b3f88 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:22.470353 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-abbe81bf-7998-42f5-ba4a-5703b42b3f88 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:22.470852 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-abbe81bf-7998-42f5-ba4a-5703b42b3f88 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:22.471629 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-abbe81bf-7998-42f5-ba4a-5703b42b3f88 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:22.472146 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-abbe81bf-7998-42f5-ba4a-5703b42b3f88 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:22.472586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-abbe81bf-7998-42f5-ba4a-5703b42b3f88 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:22.490552 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-abbe81bf-7998-42f5-ba4a-5703b42b3f88 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] HTTP exception thrown: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-abbe81bf-7998-42f5-ba4a-5703b42b3f88 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Request method failure captured: Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e HTTP/1.1 Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Skf5aWDdC40CR74ytCMLUExvGa9v-WnWSZgir8FnsqOIPgcQbm8rt7LfqPr3K-OpviTVPDVrFgjoVWp5XTqu5zvyFdSr2QLXhFcNRteklZzYEfWKkCO2Kw-We7GIm7OBNlI_jJkjuNbTKEij2gaBf7wIZ1vIvR2dikCsL6NQ3ueTQ Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Openstack-Nova-Api-Version: 2.20 Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 29c47b13a2044939b5446ffe5397633a Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:53:22.492308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:53:22.495489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/29c47b13a2044939b5446ffe5397633a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/29c47b13a2044939b5446ffe5397633a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_29c47b13a2044939b5446ffe5397633a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/29c47b13a2044939b5446ffe5397633a"}]}] Oct 28 09:53:22.495489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:22.495489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 29c47b13a2044939b5446ffe5397633a Oct 28 09:53:22.495489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-387501597 Oct 28 09:53:22.495489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-AttachVolumeShelveTestJSON-387501597-project Oct 28 09:53:22.495489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:53:22.495489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:53:22.495489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 715a782e5a56431e88e4fe5a3b0a62aa Oct 28 09:53:22.495489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-AttachVolumeShelveTestJSON-387501597-project Oct 28 09:53:22.495489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:53:22.495489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:53:22.495489 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'c716d5fd-f257-4f6a-8f52-18ad59e5c31e'} Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:53:22.497352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. Oct 28 09:53:22.500350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:53:22.503876 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-abbe81bf-7998-42f5-ba4a-5703b42b3f88 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] Returning 404 to user: Instance c716d5fd-f257-4f6a-8f52-18ad59e5c31e could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:53:22.503876 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-abbe81bf-7998-42f5-ba4a-5703b42b3f88 tempest-AttachVolumeShelveTestJSON-387501597 tempest-AttachVolumeShelveTestJSON-387501597-project] 10.222.0.25 "GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e" status: 404 len: 111 microversion: 2.20 time: 0.036788 Oct 28 09:53:22.503876 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 584/996] 10.222.0.25 () {60 vars in 1277 bytes} [Fri Oct 28 09:53:22 2022] GET /compute/v2.1/servers/c716d5fd-f257-4f6a-8f52-18ad59e5c31e => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 28 09:53:31.542387 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 28 09:53:31.543023 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Fri Oct 28 09:53:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.25) !!! Oct 28 09:53:31.698628 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6743e107-75dd-437c-a82a-0afd12aa87d7 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] Calling method '.version_select at 0x7f271a49d3a0>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:31.767814 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6743e107-75dd-437c-a82a-0afd12aa87d7 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/920a7531-7e03-4f2c-8dde-02ef6bfc6b54" status: 200 len: 350 microversion: 2.1 time: 0.072739 Oct 28 09:53:31.768708 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 585/997] 10.222.0.25 () {58 vars in 1245 bytes} [Fri Oct 28 09:53:31 2022] GET /compute/v2.1/os-volumes/920a7531-7e03-4f2c-8dde-02ef6bfc6b54 => generated 350 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:53:33.290302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:53:33.291111 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=74,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:53:33.297295 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-03cc19de-8d55-4aee-82c1-30621ff91135 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] Calling method '.version_select at 0x7f271a5024c0>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:33.362140 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-03cc19de-8d55-4aee-82c1-30621ff91135 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/9adb31dc-a39d-47ed-8163-8d23c7d42b83" status: 200 len: 351 microversion: 2.1 time: 0.072234 Oct 28 09:53:33.363269 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 586/998] 10.222.0.25 () {58 vars in 1245 bytes} [Fri Oct 28 09:53:33 2022] GET /compute/v2.1/os-volumes/9adb31dc-a39d-47ed-8163-8d23c7d42b83 => generated 351 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:53:34.893877 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-802abcaa-86ce-467b-9809-2e60d6cefe67 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] Calling method '.version_select at 0x7f271a3e34c0>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:34.951187 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-802abcaa-86ce-467b-9809-2e60d6cefe67 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/008dedd5-675d-4edb-843d-4f471f9cd95d" status: 200 len: 350 microversion: 2.1 time: 0.061510 Oct 28 09:53:34.951808 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 587/999] 10.222.0.25 () {58 vars in 1245 bytes} [Fri Oct 28 09:53:34 2022] GET /compute/v2.1/os-volumes/008dedd5-675d-4edb-843d-4f471f9cd95d => generated 350 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:53:34.965604 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1ee9c79d-86f8-4476-b197-f824be9620de tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] Calling method '.version_select at 0x7f271a489c10>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:35.059508 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1ee9c79d-86f8-4476-b197-f824be9620de tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] 10.222.0.25 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.097996 Oct 28 09:53:35.060114 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 588/1000] 10.222.0.25 () {58 vars in 1134 bytes} [Fri Oct 28 09:53:34 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:35.074809 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3fcefe4f-72eb-46fb-acc0-0fe8321f0848 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] Calling method '.version_select at 0x7f271a49dd30>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:35.171460 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3fcefe4f-72eb-46fb-acc0-0fe8321f0848 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] 10.222.0.25 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.100121 Oct 28 09:53:35.174229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 589/1001] 10.222.0.25 () {58 vars in 1149 bytes} [Fri Oct 28 09:53:35 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:53:35.190353 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7e2d39a6-6c85-47a1-b483-62d8b7168ec6 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] Calling method '.version_select at 0x7f271a502b80>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:35.289154 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7e2d39a6-6c85-47a1-b483-62d8b7168ec6 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] 10.222.0.25 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.102189 Oct 28 09:53:35.298460 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 590/1002] 10.222.0.25 () {58 vars in 1134 bytes} [Fri Oct 28 09:53:35 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:35.306519 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-05e86c75-a4b6-45c8-891a-3b35264e9aec tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] Calling method '.version_select at 0x7f271a49d670>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:35.455654 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-05e86c75-a4b6-45c8-891a-3b35264e9aec tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] 10.222.0.25 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.154727 Oct 28 09:53:35.456251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 591/1003] 10.222.0.25 () {58 vars in 1167 bytes} [Fri Oct 28 09:53:35 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:53:35.471928 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-494f84aa-171e-44d1-9ef7-4c1226af1ef3 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] Calling method '.version_select at 0x7f271dd3caf0>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:35.589704 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-494f84aa-171e-44d1-9ef7-4c1226af1ef3 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.121550 Oct 28 09:53:35.590656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 592/1004] 10.222.0.25 () {58 vars in 1170 bytes} [Fri Oct 28 09:53:35 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:53:35.609846 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2a926ec9-1375-4ce8-8bd1-b1b02474b914 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] Calling method '.version_select at 0x7f271a581ca0>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:35.698916 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2a926ec9-1375-4ce8-8bd1-b1b02474b914 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.095656 Oct 28 09:53:35.699775 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 593/1005] 10.222.0.25 () {58 vars in 1155 bytes} [Fri Oct 28 09:53:35 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:35.721447 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7fdda7c3-5a3b-499f-bc1a-92e1072019b4 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] Calling method '.version_select at 0x7f271a489c10>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:35.811970 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7fdda7c3-5a3b-499f-bc1a-92e1072019b4 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.094249 Oct 28 09:53:35.812998 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 594/1006] 10.222.0.25 () {58 vars in 1188 bytes} [Fri Oct 28 09:53:35 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:53:35.830895 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3aa9fd1a-deef-4ad9-a2f3-56d66b058789 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] Calling method '.version_select at 0x7f271a49d550>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:35.926257 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3aa9fd1a-deef-4ad9-a2f3-56d66b058789 tempest-VolumesTestJSON-1635636656 tempest-VolumesTestJSON-1635636656-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.102531 Oct 28 09:53:35.927291 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 595/1007] 10.222.0.25 () {58 vars in 1155 bytes} [Fri Oct 28 09:53:35 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:49.684091 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-830947049", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "networks": [{"uuid": "c1e4415f-14a7-48f8-b5f3-cad04c5a253b"}]}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:53:49.761448 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.network.neutron [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] validate_networks() for [('c1e4415f-14a7-48f8-b5f3-cad04c5a253b', None, None, None, None, None)] {{(pid=127030) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:53:50.521046 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Flavor limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:53:50.521421 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Image limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:53:50.521885 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Flavor pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:53:50.522286 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Image pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:53:50.522716 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:53:50.532988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:50.533515 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:50.534001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:50.534816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:50.535311 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:50.535867 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:50.557711 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Getting quotas for project 6737c612807a468797742417d59da976. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:53:50.562837 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Getting quotas for user a3c6fa6f402143b598108c6cb2aedf78 and project 6737c612807a468797742417d59da976. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:53:50.575988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Going to run 1 instances... {{(pid=127030) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:53:50.578337 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127030) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:53:50.629295 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8e20d098-6f1a-4551-83a6-5a2c8c181d36 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.950278 Oct 28 09:53:50.630071 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 596/1008] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:53:49 2022] POST /compute/v2.1/servers => generated 380 bytes in 952 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 28 09:53:50.641732 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-330bcb4a-74b8-4c2b-8599-67d75e2b3620 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:50.642249 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-330bcb4a-74b8-4c2b-8599-67d75e2b3620 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:50.702010 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-330bcb4a-74b8-4c2b-8599-67d75e2b3620 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:50.709636 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-330bcb4a-74b8-4c2b-8599-67d75e2b3620 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1213 microversion: 2.1 time: 0.071964 Oct 28 09:53:50.710227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 597/1009] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:50 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1213 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:51.155934 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 28 09:53:51.156595 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Fri Oct 28 09:53:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.25) !!! Oct 28 09:53:51.734981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-59bd7095-9a3c-4db7-9ff6-f075edd06e95 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:51.735888 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-59bd7095-9a3c-4db7-9ff6-f075edd06e95 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:51.743004 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59bd7095-9a3c-4db7-9ff6-f075edd06e95 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:51.743494 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59bd7095-9a3c-4db7-9ff6-f075edd06e95 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:51.743922 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59bd7095-9a3c-4db7-9ff6-f075edd06e95 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:51.744674 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59bd7095-9a3c-4db7-9ff6-f075edd06e95 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:51.745122 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59bd7095-9a3c-4db7-9ff6-f075edd06e95 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:51.745544 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-59bd7095-9a3c-4db7-9ff6-f075edd06e95 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:51.817262 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-59bd7095-9a3c-4db7-9ff6-f075edd06e95 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:51.828466 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-59bd7095-9a3c-4db7-9ff6-f075edd06e95 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1265 microversion: 2.1 time: 0.098779 Oct 28 09:53:51.829066 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 598/1010] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:51 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1265 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:52.860939 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7359a6ce-2ec2-4da2-a2fa-95c194088c5f tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:52.860939 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7359a6ce-2ec2-4da2-a2fa-95c194088c5f tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:52.871764 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7359a6ce-2ec2-4da2-a2fa-95c194088c5f tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:52.872869 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7359a6ce-2ec2-4da2-a2fa-95c194088c5f tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:52.873847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7359a6ce-2ec2-4da2-a2fa-95c194088c5f tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:52.875105 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7359a6ce-2ec2-4da2-a2fa-95c194088c5f tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:52.875647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7359a6ce-2ec2-4da2-a2fa-95c194088c5f tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:52.876141 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7359a6ce-2ec2-4da2-a2fa-95c194088c5f tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:53.124310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7359a6ce-2ec2-4da2-a2fa-95c194088c5f tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:53.133672 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7359a6ce-2ec2-4da2-a2fa-95c194088c5f tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1313 microversion: 2.1 time: 0.281969 Oct 28 09:53:53.134596 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 599/1011] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:52 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1313 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:54.092905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-d1a4e5b0-72c8-46d2-aa52-5da89efbef1b req-d426b892-d089-4e63-a4dc-2e06f5258cf1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0aa0241a-cd52-429f-b1f3-16194c5e029c", "tag": "0ddbc0c0-ff11-4444-9779-f95e0cbe36ad"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:53:54.123416 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-d1a4e5b0-72c8-46d2-aa52-5da89efbef1b req-d426b892-d089-4e63-a4dc-2e06f5258cf1 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:54.124523 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-d1a4e5b0-72c8-46d2-aa52-5da89efbef1b req-d426b892-d089-4e63-a4dc-2e06f5258cf1 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:54.125453 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-d1a4e5b0-72c8-46d2-aa52-5da89efbef1b req-d426b892-d089-4e63-a4dc-2e06f5258cf1 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:54.149371 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-d1a4e5b0-72c8-46d2-aa52-5da89efbef1b req-d426b892-d089-4e63-a4dc-2e06f5258cf1 service nova] Creating event network-changed:0ddbc0c0-ff11-4444-9779-f95e0cbe36ad for instance 0aa0241a-cd52-429f-b1f3-16194c5e029c on hv-cinder-zed-3 Oct 28 09:53:54.161313 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-d1a4e5b0-72c8-46d2-aa52-5da89efbef1b req-d426b892-d089-4e63-a4dc-2e06f5258cf1 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074533 Oct 28 09:53:54.166607 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 600/1012] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:53:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:53:54.207376 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-265ef62e-fbe4-40e8-8b8e-fde7078c8eb3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:54.209028 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-265ef62e-fbe4-40e8-8b8e-fde7078c8eb3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:54.221602 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-265ef62e-fbe4-40e8-8b8e-fde7078c8eb3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:54.223577 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-265ef62e-fbe4-40e8-8b8e-fde7078c8eb3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:54.224352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-265ef62e-fbe4-40e8-8b8e-fde7078c8eb3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:54.225561 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-265ef62e-fbe4-40e8-8b8e-fde7078c8eb3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:54.226356 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-265ef62e-fbe4-40e8-8b8e-fde7078c8eb3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:54.227103 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-265ef62e-fbe4-40e8-8b8e-fde7078c8eb3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:54.564493 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-265ef62e-fbe4-40e8-8b8e-fde7078c8eb3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:54.586095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-265ef62e-fbe4-40e8-8b8e-fde7078c8eb3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1313 microversion: 2.1 time: 0.415967 Oct 28 09:53:54.591932 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 601/1013] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:54 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1313 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:53:55.629392 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-caa327af-c8a2-40b5-93e0-1e4fb3df1550 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:55.630795 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-caa327af-c8a2-40b5-93e0-1e4fb3df1550 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:55.644188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-caa327af-c8a2-40b5-93e0-1e4fb3df1550 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:55.644824 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-caa327af-c8a2-40b5-93e0-1e4fb3df1550 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:55.645445 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-caa327af-c8a2-40b5-93e0-1e4fb3df1550 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:55.646626 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-caa327af-c8a2-40b5-93e0-1e4fb3df1550 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:55.647266 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-caa327af-c8a2-40b5-93e0-1e4fb3df1550 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:55.648066 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-caa327af-c8a2-40b5-93e0-1e4fb3df1550 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:55.865836 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-caa327af-c8a2-40b5-93e0-1e4fb3df1550 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:55.883854 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-caa327af-c8a2-40b5-93e0-1e4fb3df1550 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1313 microversion: 2.1 time: 0.261305 Oct 28 09:53:55.898978 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 602/1014] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:55 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1313 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:53:56.923513 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b4501221-7afa-461f-b629-15f35faec0fc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:56.934938 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b4501221-7afa-461f-b629-15f35faec0fc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:56.944981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b4501221-7afa-461f-b629-15f35faec0fc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:56.945605 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b4501221-7afa-461f-b629-15f35faec0fc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:56.946036 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b4501221-7afa-461f-b629-15f35faec0fc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:56.947023 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b4501221-7afa-461f-b629-15f35faec0fc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:56.947478 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b4501221-7afa-461f-b629-15f35faec0fc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:56.947896 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b4501221-7afa-461f-b629-15f35faec0fc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:57.184167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b4501221-7afa-461f-b629-15f35faec0fc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:57.198265 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b4501221-7afa-461f-b629-15f35faec0fc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1313 microversion: 2.1 time: 0.278999 Oct 28 09:53:57.199386 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 603/1015] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:56 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1313 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:58.236965 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-fd568c60-550d-4b04-b282-b63220e97816 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:58.237910 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-fd568c60-550d-4b04-b282-b63220e97816 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:58.250257 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd568c60-550d-4b04-b282-b63220e97816 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:58.250793 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd568c60-550d-4b04-b282-b63220e97816 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:58.251749 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd568c60-550d-4b04-b282-b63220e97816 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:58.252434 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd568c60-550d-4b04-b282-b63220e97816 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:58.253143 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd568c60-550d-4b04-b282-b63220e97816 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:58.253791 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fd568c60-550d-4b04-b282-b63220e97816 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:58.458337 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-fd568c60-550d-4b04-b282-b63220e97816 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:58.477324 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-fd568c60-550d-4b04-b282-b63220e97816 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1313 microversion: 2.1 time: 0.247971 Oct 28 09:53:58.478854 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 604/1016] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:58 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1313 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:59.521383 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b486dbb6-6ddc-40a1-84c9-5237f053775a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:53:59.521383 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b486dbb6-6ddc-40a1-84c9-5237f053775a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:53:59.529996 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b486dbb6-6ddc-40a1-84c9-5237f053775a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:59.530523 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b486dbb6-6ddc-40a1-84c9-5237f053775a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:59.530970 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b486dbb6-6ddc-40a1-84c9-5237f053775a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:59.531697 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b486dbb6-6ddc-40a1-84c9-5237f053775a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:53:59.532152 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b486dbb6-6ddc-40a1-84c9-5237f053775a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:53:59.532577 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b486dbb6-6ddc-40a1-84c9-5237f053775a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:53:59.727773 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b486dbb6-6ddc-40a1-84c9-5237f053775a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:53:59.743040 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b486dbb6-6ddc-40a1-84c9-5237f053775a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1313 microversion: 2.1 time: 0.226328 Oct 28 09:53:59.744099 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 605/1017] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:53:59 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1313 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:53:59.753184 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-790226382", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452"}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:53:59.834167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.network.neutron [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] validate_networks() for None {{(pid=127030) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:54:00.255411 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Flavor limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:54:00.256117 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Image limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:54:00.256938 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Flavor pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:54:00.257676 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Image pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:54:00.258874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:54:00.271197 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:00.271673 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:00.272113 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:00.272838 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:00.273280 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:00.273694 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:00.300630 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Getting quotas for project 1d832ea68ebb4c60b962d0cd76f787b8. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:54:00.310502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Getting quotas for user 4d088636881e45559699462268bd0411 and project 1d832ea68ebb4c60b962d0cd76f787b8. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:54:00.335377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Going to run 1 instances... {{(pid=127030) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:54:00.340075 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127030) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:54:00.410198 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-d07fb3ac-56f6-46e3-b618-c705fa6bb42d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.664350 Oct 28 09:54:00.411374 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 606/1018] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:53:59 2022] POST /compute/v2.1/servers => generated 380 bytes in 666 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 09:54:00.424209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7b9f65fb-1f3a-41f2-8b76-05ec139b7254 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:00.424822 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7b9f65fb-1f3a-41f2-8b76-05ec139b7254 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:00.518016 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7b9f65fb-1f3a-41f2-8b76-05ec139b7254 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:00.540811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7b9f65fb-1f3a-41f2-8b76-05ec139b7254 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1214 microversion: 2.1 time: 0.120330 Oct 28 09:54:00.541973 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 607/1019] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:00 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1214 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:00.780823 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f98bbdda-bdd3-4417-b79a-4841e462f23e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:00.781686 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f98bbdda-bdd3-4417-b79a-4841e462f23e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:00.792909 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f98bbdda-bdd3-4417-b79a-4841e462f23e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:00.794131 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f98bbdda-bdd3-4417-b79a-4841e462f23e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:00.796154 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f98bbdda-bdd3-4417-b79a-4841e462f23e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:00.798039 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f98bbdda-bdd3-4417-b79a-4841e462f23e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:00.799112 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f98bbdda-bdd3-4417-b79a-4841e462f23e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:00.799958 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f98bbdda-bdd3-4417-b79a-4841e462f23e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:01.096092 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f98bbdda-bdd3-4417-b79a-4841e462f23e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:01.114335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f98bbdda-bdd3-4417-b79a-4841e462f23e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1313 microversion: 2.1 time: 0.341189 Oct 28 09:54:01.116341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 608/1020] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:00 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1313 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:01.144892 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-44aa861c-2197-495d-8340-840a3cb0f7d9 req-66183d15-5ce6-4c09-8c8a-4e22b0b67d41 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aa0241a-cd52-429f-b1f3-16194c5e029c", "name": "network-vif-plugged", "status": "completed", "tag": "0ddbc0c0-ff11-4444-9779-f95e0cbe36ad"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:54:01.163825 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-44aa861c-2197-495d-8340-840a3cb0f7d9 req-66183d15-5ce6-4c09-8c8a-4e22b0b67d41 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:01.164951 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-44aa861c-2197-495d-8340-840a3cb0f7d9 req-66183d15-5ce6-4c09-8c8a-4e22b0b67d41 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:01.165718 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-44aa861c-2197-495d-8340-840a3cb0f7d9 req-66183d15-5ce6-4c09-8c8a-4e22b0b67d41 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:01.187781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-44aa861c-2197-495d-8340-840a3cb0f7d9 req-66183d15-5ce6-4c09-8c8a-4e22b0b67d41 service nova] Creating event network-vif-plugged:0ddbc0c0-ff11-4444-9779-f95e0cbe36ad for instance 0aa0241a-cd52-429f-b1f3-16194c5e029c on hv-cinder-zed-3 Oct 28 09:54:01.198431 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-44aa861c-2197-495d-8340-840a3cb0f7d9 req-66183d15-5ce6-4c09-8c8a-4e22b0b67d41 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079005 Oct 28 09:54:01.199595 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 609/1021] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:54:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:54:01.591793 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7acd0e23-9fad-4e1b-b1b4-5c95713ce466 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:01.594324 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7acd0e23-9fad-4e1b-b1b4-5c95713ce466 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:01.615555 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7acd0e23-9fad-4e1b-b1b4-5c95713ce466 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:01.617964 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7acd0e23-9fad-4e1b-b1b4-5c95713ce466 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:01.619370 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7acd0e23-9fad-4e1b-b1b4-5c95713ce466 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:01.621456 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7acd0e23-9fad-4e1b-b1b4-5c95713ce466 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:01.623753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7acd0e23-9fad-4e1b-b1b4-5c95713ce466 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:01.624839 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7acd0e23-9fad-4e1b-b1b4-5c95713ce466 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:01.705722 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7acd0e23-9fad-4e1b-b1b4-5c95713ce466 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:01.713784 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7acd0e23-9fad-4e1b-b1b4-5c95713ce466 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1266 microversion: 2.1 time: 0.136873 Oct 28 09:54:01.714376 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 610/1022] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:01 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1266 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:02.154578 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1ef62d6b-5fe2-4dc5-ab96-eecc433a7fc1 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:02.155626 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1ef62d6b-5fe2-4dc5-ab96-eecc433a7fc1 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:02.165021 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ef62d6b-5fe2-4dc5-ab96-eecc433a7fc1 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:02.165590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ef62d6b-5fe2-4dc5-ab96-eecc433a7fc1 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:02.166088 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ef62d6b-5fe2-4dc5-ab96-eecc433a7fc1 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:02.169632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ef62d6b-5fe2-4dc5-ab96-eecc433a7fc1 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:02.169632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ef62d6b-5fe2-4dc5-ab96-eecc433a7fc1 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:02.169632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1ef62d6b-5fe2-4dc5-ab96-eecc433a7fc1 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:02.361387 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1ef62d6b-5fe2-4dc5-ab96-eecc433a7fc1 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:02.369503 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1ef62d6b-5fe2-4dc5-ab96-eecc433a7fc1 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1313 microversion: 2.1 time: 0.221375 Oct 28 09:54:02.370323 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 611/1023] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:02 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1313 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:02.738884 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0ecc28ed-24b6-429d-a492-59c681cc20f8 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:02.741301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0ecc28ed-24b6-429d-a492-59c681cc20f8 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:02.752283 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0ecc28ed-24b6-429d-a492-59c681cc20f8 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:02.753396 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0ecc28ed-24b6-429d-a492-59c681cc20f8 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:02.754184 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0ecc28ed-24b6-429d-a492-59c681cc20f8 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:02.755521 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0ecc28ed-24b6-429d-a492-59c681cc20f8 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:02.756350 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0ecc28ed-24b6-429d-a492-59c681cc20f8 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:02.757119 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0ecc28ed-24b6-429d-a492-59c681cc20f8 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:02.900740 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0ecc28ed-24b6-429d-a492-59c681cc20f8 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:02.911853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0ecc28ed-24b6-429d-a492-59c681cc20f8 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1266 microversion: 2.1 time: 0.177094 Oct 28 09:54:02.912856 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 612/1024] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:02 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1266 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:03.395306 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-24f96e54-d1f6-44ca-baf8-8b8d8f0751dd tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:03.396342 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-24f96e54-d1f6-44ca-baf8-8b8d8f0751dd tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:03.406689 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24f96e54-d1f6-44ca-baf8-8b8d8f0751dd tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:03.407426 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24f96e54-d1f6-44ca-baf8-8b8d8f0751dd tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:03.408149 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24f96e54-d1f6-44ca-baf8-8b8d8f0751dd tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:03.409308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24f96e54-d1f6-44ca-baf8-8b8d8f0751dd tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:03.410030 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24f96e54-d1f6-44ca-baf8-8b8d8f0751dd tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:03.410547 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-24f96e54-d1f6-44ca-baf8-8b8d8f0751dd tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:03.829618 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-24f96e54-d1f6-44ca-baf8-8b8d8f0751dd tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:03.851388 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-24f96e54-d1f6-44ca-baf8-8b8d8f0751dd tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1497 microversion: 2.1 time: 0.460994 Oct 28 09:54:03.852516 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 613/1025] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:03 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1497 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:03.937171 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-1f1c5fbb-d865-4376-a041-352593e1f93d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:03.937793 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-1f1c5fbb-d865-4376-a041-352593e1f93d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:03.945642 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1f1c5fbb-d865-4376-a041-352593e1f93d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:03.946154 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1f1c5fbb-d865-4376-a041-352593e1f93d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:03.946752 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1f1c5fbb-d865-4376-a041-352593e1f93d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:03.948084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1f1c5fbb-d865-4376-a041-352593e1f93d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:03.948751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1f1c5fbb-d865-4376-a041-352593e1f93d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:03.949170 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-1f1c5fbb-d865-4376-a041-352593e1f93d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:04.072347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-1f1c5fbb-d865-4376-a041-352593e1f93d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:04.082756 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-1f1c5fbb-d865-4376-a041-352593e1f93d tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1272 microversion: 2.1 time: 0.150242 Oct 28 09:54:04.083347 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 614/1026] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:03 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1272 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:04.542602 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-025a13e8-d3ce-40cf-8bff-85ffdae5e94e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:04.544382 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-025a13e8-d3ce-40cf-8bff-85ffdae5e94e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:04.557318 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-025a13e8-d3ce-40cf-8bff-85ffdae5e94e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:04.558379 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-025a13e8-d3ce-40cf-8bff-85ffdae5e94e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:04.559922 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-025a13e8-d3ce-40cf-8bff-85ffdae5e94e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:04.561377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-025a13e8-d3ce-40cf-8bff-85ffdae5e94e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:04.562849 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-025a13e8-d3ce-40cf-8bff-85ffdae5e94e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:04.564030 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-025a13e8-d3ce-40cf-8bff-85ffdae5e94e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:04.615891 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-025a13e8-d3ce-40cf-8bff-85ffdae5e94e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Going to try to terminate instance {{(pid=127030) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:54:04.688567 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:04.730856 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-025a13e8-d3ce-40cf-8bff-85ffdae5e94e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "DELETE /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 204 len: 0 microversion: 2.1 time: 0.194509 Oct 28 09:54:04.732405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 615/1027] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:54:04 2022] DELETE /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 28 09:54:04.742111 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-bd2c39ef-cde0-4a51-8c08-f46e73cdacbe tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:04.742874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-bd2c39ef-cde0-4a51-8c08-f46e73cdacbe tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:04.749479 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bd2c39ef-cde0-4a51-8c08-f46e73cdacbe tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:04.749972 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bd2c39ef-cde0-4a51-8c08-f46e73cdacbe tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:04.750445 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bd2c39ef-cde0-4a51-8c08-f46e73cdacbe tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:04.751184 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bd2c39ef-cde0-4a51-8c08-f46e73cdacbe tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:04.751642 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bd2c39ef-cde0-4a51-8c08-f46e73cdacbe tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:04.752069 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bd2c39ef-cde0-4a51-8c08-f46e73cdacbe tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:05.072605 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-bd2c39ef-cde0-4a51-8c08-f46e73cdacbe tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:05.083361 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-bd2c39ef-cde0-4a51-8c08-f46e73cdacbe tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1503 microversion: 2.1 time: 0.345657 Oct 28 09:54:05.084181 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 616/1028] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:04 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1503 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:05.118585 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-f63b7a2c-b99b-4306-b861-033eb5e3d7bc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:05.119802 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-f63b7a2c-b99b-4306-b861-033eb5e3d7bc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:05.131958 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f63b7a2c-b99b-4306-b861-033eb5e3d7bc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:05.132714 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f63b7a2c-b99b-4306-b861-033eb5e3d7bc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:05.133435 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f63b7a2c-b99b-4306-b861-033eb5e3d7bc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:05.134301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f63b7a2c-b99b-4306-b861-033eb5e3d7bc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:05.134912 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f63b7a2c-b99b-4306-b861-033eb5e3d7bc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:05.135665 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-f63b7a2c-b99b-4306-b861-033eb5e3d7bc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:05.761420 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-f63b7a2c-b99b-4306-b861-033eb5e3d7bc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:05.771783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-f63b7a2c-b99b-4306-b861-033eb5e3d7bc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1314 microversion: 2.1 time: 0.661368 Oct 28 09:54:05.772373 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 617/1029] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:05 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1314 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:06.117296 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ff756f51-becb-471e-acd1-698b349a9b9c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:06.118526 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ff756f51-becb-471e-acd1-698b349a9b9c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:06.129962 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ff756f51-becb-471e-acd1-698b349a9b9c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:06.130832 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ff756f51-becb-471e-acd1-698b349a9b9c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:06.131381 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ff756f51-becb-471e-acd1-698b349a9b9c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:06.132288 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ff756f51-becb-471e-acd1-698b349a9b9c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:06.132996 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ff756f51-becb-471e-acd1-698b349a9b9c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:06.133743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ff756f51-becb-471e-acd1-698b349a9b9c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:06.393961 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ff756f51-becb-471e-acd1-698b349a9b9c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:06.409411 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ff756f51-becb-471e-acd1-698b349a9b9c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1503 microversion: 2.1 time: 0.299484 Oct 28 09:54:06.410424 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 618/1030] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:06 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1503 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:06.811662 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4b107ec8-ac24-433c-a07b-402b8a65e4db tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:06.812168 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4b107ec8-ac24-433c-a07b-402b8a65e4db tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:06.819160 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b107ec8-ac24-433c-a07b-402b8a65e4db tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:06.819709 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b107ec8-ac24-433c-a07b-402b8a65e4db tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:06.820153 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b107ec8-ac24-433c-a07b-402b8a65e4db tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:06.820892 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b107ec8-ac24-433c-a07b-402b8a65e4db tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:06.821354 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b107ec8-ac24-433c-a07b-402b8a65e4db tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:06.821810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4b107ec8-ac24-433c-a07b-402b8a65e4db tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:07.203994 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-4b107ec8-ac24-433c-a07b-402b8a65e4db tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:07.219530 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4b107ec8-ac24-433c-a07b-402b8a65e4db tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1314 microversion: 2.1 time: 0.412352 Oct 28 09:54:07.221578 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 619/1031] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:06 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1314 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:07.455182 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b177654e-82cd-4f1a-a8fc-865f0935d8ce tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:07.457002 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b177654e-82cd-4f1a-a8fc-865f0935d8ce tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:07.470681 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b177654e-82cd-4f1a-a8fc-865f0935d8ce tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:07.471567 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b177654e-82cd-4f1a-a8fc-865f0935d8ce tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:07.472370 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b177654e-82cd-4f1a-a8fc-865f0935d8ce tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:07.474176 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b177654e-82cd-4f1a-a8fc-865f0935d8ce tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:07.475064 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b177654e-82cd-4f1a-a8fc-865f0935d8ce tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:07.476170 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b177654e-82cd-4f1a-a8fc-865f0935d8ce tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:07.662501 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b177654e-82cd-4f1a-a8fc-865f0935d8ce tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:07.676389 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b177654e-82cd-4f1a-a8fc-865f0935d8ce tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1461 microversion: 2.1 time: 0.228605 Oct 28 09:54:07.678446 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 620/1032] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:07 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1461 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:07.687765 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-12139fbf-2fa3-4695-b6d9-cc1cc51f3036 req-7b6f988c-8015-405c-a7df-b603fb47f02b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3", "tag": "ad6ab887-bf1b-4af1-98a6-d679a90f2ddb"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:54:07.708520 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-12139fbf-2fa3-4695-b6d9-cc1cc51f3036 req-7b6f988c-8015-405c-a7df-b603fb47f02b service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:07.709698 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-12139fbf-2fa3-4695-b6d9-cc1cc51f3036 req-7b6f988c-8015-405c-a7df-b603fb47f02b service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:07.710796 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-12139fbf-2fa3-4695-b6d9-cc1cc51f3036 req-7b6f988c-8015-405c-a7df-b603fb47f02b service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:07.743758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-12139fbf-2fa3-4695-b6d9-cc1cc51f3036 req-7b6f988c-8015-405c-a7df-b603fb47f02b service nova] Creating event network-changed:ad6ab887-bf1b-4af1-98a6-d679a90f2ddb for instance 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 on hv-cinder-zed-3 Oct 28 09:54:07.755311 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-12139fbf-2fa3-4695-b6d9-cc1cc51f3036 req-7b6f988c-8015-405c-a7df-b603fb47f02b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074229 Oct 28 09:54:07.756424 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 621/1033] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:54:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:54:08.259181 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8b16b982-692e-4054-adb8-a5356516efbf tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:08.259987 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8b16b982-692e-4054-adb8-a5356516efbf tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:08.268283 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8b16b982-692e-4054-adb8-a5356516efbf tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:08.268770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8b16b982-692e-4054-adb8-a5356516efbf tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:08.269203 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8b16b982-692e-4054-adb8-a5356516efbf tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:08.269918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8b16b982-692e-4054-adb8-a5356516efbf tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:08.270372 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8b16b982-692e-4054-adb8-a5356516efbf tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:08.270816 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8b16b982-692e-4054-adb8-a5356516efbf tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:08.442654 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8b16b982-692e-4054-adb8-a5356516efbf tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:08.456892 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8b16b982-692e-4054-adb8-a5356516efbf tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1314 microversion: 2.1 time: 0.202123 Oct 28 09:54:08.471274 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 622/1034] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:08 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1314 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:08.721643 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:08.722787 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:08.733539 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:08.734635 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:08.735022 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:08.735898 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:08.736481 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:08.737001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:08.780506 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:08.780867 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:08.781316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:08.781882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lazy-loading 'fault' on Instance uuid 0aa0241a-cd52-429f-b1f3-16194c5e029c {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:54:08.847867 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:08.861873 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-e84a9922-47c1-4c72-84c8-402753df318a tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 200 len: 1303 microversion: 2.1 time: 0.144793 Oct 28 09:54:08.862848 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 623/1035] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:08 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 1303 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:09.496203 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b0c346c6-1e5b-4a4f-b91a-ae10ddc7acbc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:09.503813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b0c346c6-1e5b-4a4f-b91a-ae10ddc7acbc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:09.524939 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b0c346c6-1e5b-4a4f-b91a-ae10ddc7acbc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:09.525912 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b0c346c6-1e5b-4a4f-b91a-ae10ddc7acbc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:09.526928 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b0c346c6-1e5b-4a4f-b91a-ae10ddc7acbc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:09.528434 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b0c346c6-1e5b-4a4f-b91a-ae10ddc7acbc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:09.529377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b0c346c6-1e5b-4a4f-b91a-ae10ddc7acbc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:09.530207 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b0c346c6-1e5b-4a4f-b91a-ae10ddc7acbc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:09.849138 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b0c346c6-1e5b-4a4f-b91a-ae10ddc7acbc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:09.859475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b0c346c6-1e5b-4a4f-b91a-ae10ddc7acbc tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1314 microversion: 2.1 time: 0.377199 Oct 28 09:54:09.860209 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 624/1036] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:09 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1314 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:09.866129 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-ddbaf5fc-256c-4d90-a41a-02794e684237 req-4d4adb96-c675-4295-93ec-372ad5852e4e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aa0241a-cd52-429f-b1f3-16194c5e029c", "name": "network-vif-deleted", "tag": "0ddbc0c0-ff11-4444-9779-f95e0cbe36ad"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:54:09.874306 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:09.880792 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-ddbaf5fc-256c-4d90-a41a-02794e684237 req-4d4adb96-c675-4295-93ec-372ad5852e4e service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:09.881569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-ddbaf5fc-256c-4d90-a41a-02794e684237 req-4d4adb96-c675-4295-93ec-372ad5852e4e service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:09.882138 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-ddbaf5fc-256c-4d90-a41a-02794e684237 req-4d4adb96-c675-4295-93ec-372ad5852e4e service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:09.900811 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-ddbaf5fc-256c-4d90-a41a-02794e684237 req-4d4adb96-c675-4295-93ec-372ad5852e4e service nova] Creating event network-vif-deleted:0ddbc0c0-ff11-4444-9779-f95e0cbe36ad for instance 0aa0241a-cd52-429f-b1f3-16194c5e029c on hv-cinder-zed-3 Oct 28 09:54:09.907758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-ddbaf5fc-256c-4d90-a41a-02794e684237 req-4d4adb96-c675-4295-93ec-372ad5852e4e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045991 Oct 28 09:54:09.908852 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 625/1037] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:54:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:54:09.916505 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3d3fdb73-f208-4b99-8be2-eb9e6f424853 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:09.917390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3d3fdb73-f208-4b99-8be2-eb9e6f424853 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 0aa0241a-cd52-429f-b1f3-16194c5e029c] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:09.927182 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3d3fdb73-f208-4b99-8be2-eb9e6f424853 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:09.927863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3d3fdb73-f208-4b99-8be2-eb9e6f424853 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:09.928397 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3d3fdb73-f208-4b99-8be2-eb9e6f424853 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:09.929237 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3d3fdb73-f208-4b99-8be2-eb9e6f424853 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:09.929993 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3d3fdb73-f208-4b99-8be2-eb9e6f424853 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:09.930743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3d3fdb73-f208-4b99-8be2-eb9e6f424853 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:09.947714 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-3d3fdb73-f208-4b99-8be2-eb9e6f424853 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] HTTP exception thrown: Instance 0aa0241a-cd52-429f-b1f3-16194c5e029c could not be found. Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3d3fdb73-f208-4b99-8be2-eb9e6f424853 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Request method failure captured: Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c HTTP/1.1 Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6apzCi3qRWmIc52YfW5_muRZHyojKioFgPYDZZF22VzJ9G120yqUWTXaPaNW0CLEahGYHec3odWY-4qUYzAgdcuGH57jZpcLrj82y8oKpOFJV8odT0HMRXfmxBs5SDtTn2laSd6k9RWOUid3VGokRZ0CEt7mJcKO8KlV_7jV3PE_wA Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 6737c612807a468797742417d59da976 Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-VolumesActionsTest-1640912710 Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:54:09.949656 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:54:09.953980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/6737c612807a468797742417d59da976"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6737c612807a468797742417d59da976"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_6737c612807a468797742417d59da976"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6737c612807a468797742417d59da976"}]}] Oct 28 09:54:09.953980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-VolumesActionsTest-1640912710 Oct 28 09:54:09.953980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 6737c612807a468797742417d59da976 Oct 28 09:54:09.953980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-VolumesActionsTest-1640912710 Oct 28 09:54:09.953980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-VolumesActionsTest-1640912710-project Oct 28 09:54:09.953980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:54:09.953980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:54:09.953980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: a3c6fa6f402143b598108c6cb2aedf78 Oct 28 09:54:09.953980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-VolumesActionsTest-1640912710-project Oct 28 09:54:09.953980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:54:09.953980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 0aa0241a-cd52-429f-b1f3-16194c5e029c could not be found. Oct 28 09:54:09.953980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '0aa0241a-cd52-429f-b1f3-16194c5e029c'} Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0aa0241a-cd52-429f-b1f3-16194c5e029c could not be found. Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:54:09.955723 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0aa0241a-cd52-429f-b1f3-16194c5e029c could not be found. Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0aa0241a-cd52-429f-b1f3-16194c5e029c could not be found. Oct 28 09:54:09.958287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:09.960879 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3d3fdb73-f208-4b99-8be2-eb9e6f424853 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Returning 404 to user: Instance 0aa0241a-cd52-429f-b1f3-16194c5e029c could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:54:09.960879 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3d3fdb73-f208-4b99-8be2-eb9e6f424853 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c" status: 404 len: 111 microversion: 2.1 time: 0.040818 Oct 28 09:54:09.960879 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 626/1038] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:09 2022] GET /compute/v2.1/servers/0aa0241a-cd52-429f-b1f3-16194c5e029c => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:54:10.119782 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1237150551", "imageRef": "506a75ac-7d29-4cae-8832-e8cce9eafe97", "flavorRef": "452", "networks": [{"uuid": "c1e4415f-14a7-48f8-b5f3-cad04c5a253b"}]}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:54:10.182548 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.network.neutron [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] validate_networks() for [('c1e4415f-14a7-48f8-b5f3-cad04c5a253b', None, None, None, None, None)] {{(pid=127030) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 09:54:10.282642 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=4 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:10.529518 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=16 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:10.718111 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Flavor limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 09:54:10.718743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Image limits 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 09:54:10.719317 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Flavor pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 09:54:10.719883 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Image pref 0:0:0 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 09:54:10.720419 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.virt.hardware [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127030) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 09:54:10.733377 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:10.733870 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:10.734307 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:10.735131 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:10.735588 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:10.736015 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:10.758954 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Getting quotas for project 6737c612807a468797742417d59da976. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 09:54:10.765711 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.quota [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Getting quotas for user a3c6fa6f402143b598108c6cb2aedf78 and project 6737c612807a468797742417d59da976. Resources: {'instances', 'ram', 'cores'} {{(pid=127030) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 09:54:10.781208 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Going to run 1 instances... {{(pid=127030) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 09:54:10.784152 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='506a75ac-7d29-4cae-8832-e8cce9eafe97',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127030) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 09:54:10.846882 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-030c808b-69e1-464a-83cf-7d666032a502 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.732634 Oct 28 09:54:10.857682 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 627/1039] 10.222.0.25 () {60 vars in 1147 bytes} [Fri Oct 28 09:54:10 2022] POST /compute/v2.1/servers => generated 380 bytes in 736 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 09:54:10.875151 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8a948eac-17e8-4ebf-b6af-56ac7098d9e8 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:10.876437 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8a948eac-17e8-4ebf-b6af-56ac7098d9e8 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:11.032849 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8a948eac-17e8-4ebf-b6af-56ac7098d9e8 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:11.055410 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8a948eac-17e8-4ebf-b6af-56ac7098d9e8 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1214 microversion: 2.1 time: 0.190314 Oct 28 09:54:11.074637 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 628/1040] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:10 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1214 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:11.074637 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9c21de3c-bbcd-47e3-9392-1a067b994bce tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:11.074637 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9c21de3c-bbcd-47e3-9392-1a067b994bce tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:11.087299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9c21de3c-bbcd-47e3-9392-1a067b994bce tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:11.088363 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9c21de3c-bbcd-47e3-9392-1a067b994bce tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:11.089240 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9c21de3c-bbcd-47e3-9392-1a067b994bce tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:11.090635 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9c21de3c-bbcd-47e3-9392-1a067b994bce tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:11.091473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9c21de3c-bbcd-47e3-9392-1a067b994bce tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:11.092302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9c21de3c-bbcd-47e3-9392-1a067b994bce tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:11.490719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-9c21de3c-bbcd-47e3-9392-1a067b994bce tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:11.499742 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9c21de3c-bbcd-47e3-9392-1a067b994bce tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1314 microversion: 2.1 time: 0.441043 Oct 28 09:54:11.504047 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 629/1041] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:11 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1314 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:12.103691 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-417c8d65-dfd9-4eb9-899e-00809acc7ecc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:12.104869 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-417c8d65-dfd9-4eb9-899e-00809acc7ecc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:12.116916 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-417c8d65-dfd9-4eb9-899e-00809acc7ecc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:12.117758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-417c8d65-dfd9-4eb9-899e-00809acc7ecc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:12.118572 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-417c8d65-dfd9-4eb9-899e-00809acc7ecc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:12.119848 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-417c8d65-dfd9-4eb9-899e-00809acc7ecc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:12.120700 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-417c8d65-dfd9-4eb9-899e-00809acc7ecc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:12.121475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-417c8d65-dfd9-4eb9-899e-00809acc7ecc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:12.218900 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-417c8d65-dfd9-4eb9-899e-00809acc7ecc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:12.233556 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-417c8d65-dfd9-4eb9-899e-00809acc7ecc tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1266 microversion: 2.1 time: 0.137222 Oct 28 09:54:12.234714 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 630/1042] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:12 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1266 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:12.545976 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b6b704c8-89d9-40fb-aae6-7793821346b1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:12.549213 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b6b704c8-89d9-40fb-aae6-7793821346b1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:12.578051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6b704c8-89d9-40fb-aae6-7793821346b1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:12.579312 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6b704c8-89d9-40fb-aae6-7793821346b1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:12.580783 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6b704c8-89d9-40fb-aae6-7793821346b1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:12.584178 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6b704c8-89d9-40fb-aae6-7793821346b1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:12.585694 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6b704c8-89d9-40fb-aae6-7793821346b1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:12.587156 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b6b704c8-89d9-40fb-aae6-7793821346b1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:13.085788 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b6b704c8-89d9-40fb-aae6-7793821346b1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:13.097207 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b6b704c8-89d9-40fb-aae6-7793821346b1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1314 microversion: 2.1 time: 0.575145 Oct 28 09:54:13.098081 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 631/1043] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:12 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1314 bytes in 577 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:13.261638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4ffd963f-3c6d-4ace-a5f8-ac5be25f6806 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:13.263796 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4ffd963f-3c6d-4ace-a5f8-ac5be25f6806 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:13.272308 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4ffd963f-3c6d-4ace-a5f8-ac5be25f6806 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:13.273914 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4ffd963f-3c6d-4ace-a5f8-ac5be25f6806 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:13.274646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4ffd963f-3c6d-4ace-a5f8-ac5be25f6806 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:13.275629 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4ffd963f-3c6d-4ace-a5f8-ac5be25f6806 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:13.276216 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4ffd963f-3c6d-4ace-a5f8-ac5be25f6806 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:13.276755 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4ffd963f-3c6d-4ace-a5f8-ac5be25f6806 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:13.422084 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-4ffd963f-3c6d-4ace-a5f8-ac5be25f6806 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:13.436911 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4ffd963f-3c6d-4ace-a5f8-ac5be25f6806 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1272 microversion: 2.1 time: 0.179471 Oct 28 09:54:13.438425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 632/1044] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:13 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1272 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:14.140464 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9b3e1981-93d0-40be-a8ae-8d34160ad1c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:14.141550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9b3e1981-93d0-40be-a8ae-8d34160ad1c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:14.151304 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9b3e1981-93d0-40be-a8ae-8d34160ad1c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:14.152407 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9b3e1981-93d0-40be-a8ae-8d34160ad1c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:14.153094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9b3e1981-93d0-40be-a8ae-8d34160ad1c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:14.154280 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9b3e1981-93d0-40be-a8ae-8d34160ad1c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:14.155513 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9b3e1981-93d0-40be-a8ae-8d34160ad1c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:14.156243 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9b3e1981-93d0-40be-a8ae-8d34160ad1c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:14.550819 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-9b3e1981-93d0-40be-a8ae-8d34160ad1c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:14.567322 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9b3e1981-93d0-40be-a8ae-8d34160ad1c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1314 microversion: 2.1 time: 0.432807 Oct 28 09:54:14.568517 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 633/1045] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:14 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1314 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:14.592046 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7d6f8bc8-2bd9-4591-8381-a41ba4643d72 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:14.600304 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7d6f8bc8-2bd9-4591-8381-a41ba4643d72 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:14.620902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7d6f8bc8-2bd9-4591-8381-a41ba4643d72 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:14.622955 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7d6f8bc8-2bd9-4591-8381-a41ba4643d72 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:14.624140 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7d6f8bc8-2bd9-4591-8381-a41ba4643d72 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:14.627775 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7d6f8bc8-2bd9-4591-8381-a41ba4643d72 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:14.628973 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7d6f8bc8-2bd9-4591-8381-a41ba4643d72 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:14.630074 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7d6f8bc8-2bd9-4591-8381-a41ba4643d72 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:14.914501 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7d6f8bc8-2bd9-4591-8381-a41ba4643d72 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:14.946689 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7d6f8bc8-2bd9-4591-8381-a41ba4643d72 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1314 microversion: 2.1 time: 0.368487 Oct 28 09:54:14.948367 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 634/1046] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:14 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1314 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:15.003951 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-5541dd11-a0ec-4074-901e-176d0481efc2 req-a790fba5-f07f-4407-b9ea-2fe9ab666722 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3", "name": "network-vif-plugged", "status": "completed", "tag": "ad6ab887-bf1b-4af1-98a6-d679a90f2ddb"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:54:15.042019 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-5541dd11-a0ec-4074-901e-176d0481efc2 req-a790fba5-f07f-4407-b9ea-2fe9ab666722 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:15.042635 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-5541dd11-a0ec-4074-901e-176d0481efc2 req-a790fba5-f07f-4407-b9ea-2fe9ab666722 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:15.046591 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-5541dd11-a0ec-4074-901e-176d0481efc2 req-a790fba5-f07f-4407-b9ea-2fe9ab666722 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:15.071049 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-5541dd11-a0ec-4074-901e-176d0481efc2 req-a790fba5-f07f-4407-b9ea-2fe9ab666722 service nova] Creating event network-vif-plugged:ad6ab887-bf1b-4af1-98a6-d679a90f2ddb for instance 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 on hv-cinder-zed-3 Oct 28 09:54:15.080478 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-5541dd11-a0ec-4074-901e-176d0481efc2 req-a790fba5-f07f-4407-b9ea-2fe9ab666722 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.105354 Oct 28 09:54:15.082017 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 635/1047] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:54:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:54:15.521048 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-cac96b3b-3963-48a1-9119-e8e92b38d3d2 req-73edf1eb-332a-4083-9096-69638e20cf56 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "352678cc-069f-4a32-b31b-7674427bbcb1", "tag": "efdf229e-e455-4624-b75f-3586f20bef0d"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:54:15.551278 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-cac96b3b-3963-48a1-9119-e8e92b38d3d2 req-73edf1eb-332a-4083-9096-69638e20cf56 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:15.552264 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-cac96b3b-3963-48a1-9119-e8e92b38d3d2 req-73edf1eb-332a-4083-9096-69638e20cf56 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:15.553255 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-cac96b3b-3963-48a1-9119-e8e92b38d3d2 req-73edf1eb-332a-4083-9096-69638e20cf56 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:15.561678 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:15.578026 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-cac96b3b-3963-48a1-9119-e8e92b38d3d2 req-73edf1eb-332a-4083-9096-69638e20cf56 service nova] Creating event network-changed:efdf229e-e455-4624-b75f-3586f20bef0d for instance 352678cc-069f-4a32-b31b-7674427bbcb1 on hv-cinder-zed-3 Oct 28 09:54:15.589772 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-cac96b3b-3963-48a1-9119-e8e92b38d3d2 req-73edf1eb-332a-4083-9096-69638e20cf56 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.078786 Oct 28 09:54:15.591633 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 636/1048] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:54:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:54:15.618718 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-0d265b66-d117-4e0c-99c0-e33bf420ac77 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:15.620152 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-0d265b66-d117-4e0c-99c0-e33bf420ac77 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:15.632159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0d265b66-d117-4e0c-99c0-e33bf420ac77 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:15.632159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0d265b66-d117-4e0c-99c0-e33bf420ac77 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:15.632609 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0d265b66-d117-4e0c-99c0-e33bf420ac77 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:15.633158 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0d265b66-d117-4e0c-99c0-e33bf420ac77 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:15.633620 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0d265b66-d117-4e0c-99c0-e33bf420ac77 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:15.634077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-0d265b66-d117-4e0c-99c0-e33bf420ac77 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:15.865067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-0d265b66-d117-4e0c-99c0-e33bf420ac77 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:15.874673 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-0d265b66-d117-4e0c-99c0-e33bf420ac77 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1314 microversion: 2.1 time: 0.261657 Oct 28 09:54:15.875155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 637/1049] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:15 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1314 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:15.993541 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-089fa3c0-3ba8-40bc-ae6d-4c46a0726dee tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:15.994275 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-089fa3c0-3ba8-40bc-ae6d-4c46a0726dee tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:16.004003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-089fa3c0-3ba8-40bc-ae6d-4c46a0726dee tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:16.004770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-089fa3c0-3ba8-40bc-ae6d-4c46a0726dee tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:16.005298 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-089fa3c0-3ba8-40bc-ae6d-4c46a0726dee tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:16.006166 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-089fa3c0-3ba8-40bc-ae6d-4c46a0726dee tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:16.006810 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-089fa3c0-3ba8-40bc-ae6d-4c46a0726dee tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:16.007325 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-089fa3c0-3ba8-40bc-ae6d-4c46a0726dee tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:16.215112 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-089fa3c0-3ba8-40bc-ae6d-4c46a0726dee tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:16.229094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-089fa3c0-3ba8-40bc-ae6d-4c46a0726dee tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1314 microversion: 2.1 time: 0.239794 Oct 28 09:54:16.230461 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 638/1050] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:15 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1314 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:16.900343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3a461534-9bc3-4d6c-88cd-39e6d00a8f89 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:16.901717 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3a461534-9bc3-4d6c-88cd-39e6d00a8f89 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:16.911593 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3a461534-9bc3-4d6c-88cd-39e6d00a8f89 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:16.912169 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3a461534-9bc3-4d6c-88cd-39e6d00a8f89 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:16.912779 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3a461534-9bc3-4d6c-88cd-39e6d00a8f89 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:16.913639 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3a461534-9bc3-4d6c-88cd-39e6d00a8f89 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:16.914224 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3a461534-9bc3-4d6c-88cd-39e6d00a8f89 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:16.914869 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3a461534-9bc3-4d6c-88cd-39e6d00a8f89 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:17.134147 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3a461534-9bc3-4d6c-88cd-39e6d00a8f89 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:17.150471 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3a461534-9bc3-4d6c-88cd-39e6d00a8f89 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1463 microversion: 2.1 time: 0.254487 Oct 28 09:54:17.151949 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 639/1051] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:16 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1463 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:17.207231 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-cc38c90e-f25b-44a6-9f3c-2752aa74674e tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:17.208092 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-cc38c90e-f25b-44a6-9f3c-2752aa74674e tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:17.216598 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cc38c90e-f25b-44a6-9f3c-2752aa74674e tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:17.217259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cc38c90e-f25b-44a6-9f3c-2752aa74674e tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:17.217716 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cc38c90e-f25b-44a6-9f3c-2752aa74674e tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:17.218961 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cc38c90e-f25b-44a6-9f3c-2752aa74674e tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:17.219719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cc38c90e-f25b-44a6-9f3c-2752aa74674e tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:17.220060 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cc38c90e-f25b-44a6-9f3c-2752aa74674e tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:17.253847 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-cc38c90e-f25b-44a6-9f3c-2752aa74674e tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Going to try to terminate instance {{(pid=127030) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:54:17.355081 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-cc38c90e-f25b-44a6-9f3c-2752aa74674e tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "DELETE /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 204 len: 0 microversion: 2.1 time: 0.151469 Oct 28 09:54:17.355803 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 640/1052] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:54:17 2022] DELETE /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 09:54:17.361828 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-8539b10d-92f4-475a-b161-df2b273eccde tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:17.362983 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-8539b10d-92f4-475a-b161-df2b273eccde tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:17.370592 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8539b10d-92f4-475a-b161-df2b273eccde tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:17.372019 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8539b10d-92f4-475a-b161-df2b273eccde tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:17.372515 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8539b10d-92f4-475a-b161-df2b273eccde tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:17.373551 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8539b10d-92f4-475a-b161-df2b273eccde tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:17.374162 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8539b10d-92f4-475a-b161-df2b273eccde tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:17.374671 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-8539b10d-92f4-475a-b161-df2b273eccde tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:17.538878 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-8539b10d-92f4-475a-b161-df2b273eccde tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:17.548244 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-8539b10d-92f4-475a-b161-df2b273eccde tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1314 microversion: 2.1 time: 0.190522 Oct 28 09:54:17.549126 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 641/1053] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:17 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1314 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:17.558645 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-03e502f7-6490-4d11-82cd-cc6623d402d7 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:17.560197 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-03e502f7-6490-4d11-82cd-cc6623d402d7 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:17.569846 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e502f7-6490-4d11-82cd-cc6623d402d7 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:17.570684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e502f7-6490-4d11-82cd-cc6623d402d7 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:17.571373 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e502f7-6490-4d11-82cd-cc6623d402d7 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:17.572781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e502f7-6490-4d11-82cd-cc6623d402d7 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:17.573459 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e502f7-6490-4d11-82cd-cc6623d402d7 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:17.574202 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-03e502f7-6490-4d11-82cd-cc6623d402d7 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:17.757986 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-03e502f7-6490-4d11-82cd-cc6623d402d7 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:17.767262 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-03e502f7-6490-4d11-82cd-cc6623d402d7 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1469 microversion: 2.1 time: 0.216923 Oct 28 09:54:17.767855 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 642/1054] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:17 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1469 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:18.582292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5e002633-aea9-40f8-8580-97131d0416d3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:18.583198 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5e002633-aea9-40f8-8580-97131d0416d3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:18.594753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e002633-aea9-40f8-8580-97131d0416d3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:18.595686 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e002633-aea9-40f8-8580-97131d0416d3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:18.596538 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e002633-aea9-40f8-8580-97131d0416d3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:18.597831 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e002633-aea9-40f8-8580-97131d0416d3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:18.598812 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e002633-aea9-40f8-8580-97131d0416d3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:18.599610 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5e002633-aea9-40f8-8580-97131d0416d3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:18.822782 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5e002633-aea9-40f8-8580-97131d0416d3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:18.834632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5e002633-aea9-40f8-8580-97131d0416d3 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1314 microversion: 2.1 time: 0.261856 Oct 28 09:54:18.835113 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 643/1055] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:18 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1314 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:18.845345 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-2827c2c6-7324-4bd0-bf93-531e568c8307 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:18.846499 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-2827c2c6-7324-4bd0-bf93-531e568c8307 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:18.858443 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2827c2c6-7324-4bd0-bf93-531e568c8307 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:18.859316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2827c2c6-7324-4bd0-bf93-531e568c8307 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:18.860140 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2827c2c6-7324-4bd0-bf93-531e568c8307 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:18.861424 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2827c2c6-7324-4bd0-bf93-531e568c8307 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:18.862241 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2827c2c6-7324-4bd0-bf93-531e568c8307 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:18.863046 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-2827c2c6-7324-4bd0-bf93-531e568c8307 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:19.043202 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-2827c2c6-7324-4bd0-bf93-531e568c8307 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:19.052439 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-2827c2c6-7324-4bd0-bf93-531e568c8307 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1469 microversion: 2.1 time: 0.215074 Oct 28 09:54:19.053034 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 644/1056] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:18 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1469 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:19.867957 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-98266227-321c-4268-894f-d1eef42abb2e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:19.872865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-98266227-321c-4268-894f-d1eef42abb2e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:19.886482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-98266227-321c-4268-894f-d1eef42abb2e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:19.888066 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-98266227-321c-4268-894f-d1eef42abb2e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:19.889603 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-98266227-321c-4268-894f-d1eef42abb2e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:19.890385 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-98266227-321c-4268-894f-d1eef42abb2e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:19.891337 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-98266227-321c-4268-894f-d1eef42abb2e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:19.892221 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-98266227-321c-4268-894f-d1eef42abb2e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:20.296245 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-98266227-321c-4268-894f-d1eef42abb2e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:20.310569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-98266227-321c-4268-894f-d1eef42abb2e tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1314 microversion: 2.1 time: 0.450726 Oct 28 09:54:20.312081 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 645/1057] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:19 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1314 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:20.321378 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-11c9fb98-0256-4f7a-9fdf-0367e55295c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:20.322595 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-11c9fb98-0256-4f7a-9fdf-0367e55295c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:20.349154 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11c9fb98-0256-4f7a-9fdf-0367e55295c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:20.350475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11c9fb98-0256-4f7a-9fdf-0367e55295c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:20.351368 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11c9fb98-0256-4f7a-9fdf-0367e55295c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:20.352391 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11c9fb98-0256-4f7a-9fdf-0367e55295c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:20.353159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11c9fb98-0256-4f7a-9fdf-0367e55295c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:20.353874 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-11c9fb98-0256-4f7a-9fdf-0367e55295c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:20.675611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-11c9fb98-0256-4f7a-9fdf-0367e55295c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d088636881e45559699462268bd0411', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d832ea68ebb4c60b962d0cd76f787b8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:20.696349 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-11c9fb98-0256-4f7a-9fdf-0367e55295c5 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 200 len: 1469 microversion: 2.1 time: 0.382324 Oct 28 09:54:20.697452 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 646/1058] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:20 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 1469 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:20.739699 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-4b5187a4-1549-4b02-8003-3adcaebd9e2d req-4180ad7c-796d-4f09-94e4-d372874dc496 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3", "name": "network-vif-deleted", "tag": "ad6ab887-bf1b-4af1-98a6-d679a90f2ddb"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:54:20.754586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:20.763070 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-4b5187a4-1549-4b02-8003-3adcaebd9e2d req-4180ad7c-796d-4f09-94e4-d372874dc496 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:20.764152 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-4b5187a4-1549-4b02-8003-3adcaebd9e2d req-4180ad7c-796d-4f09-94e4-d372874dc496 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:20.764850 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-4b5187a4-1549-4b02-8003-3adcaebd9e2d req-4180ad7c-796d-4f09-94e4-d372874dc496 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:20.783205 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-4b5187a4-1549-4b02-8003-3adcaebd9e2d req-4180ad7c-796d-4f09-94e4-d372874dc496 service nova] Creating event network-vif-deleted:ad6ab887-bf1b-4af1-98a6-d679a90f2ddb for instance 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 on hv-cinder-zed-3 Oct 28 09:54:20.790664 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-4b5187a4-1549-4b02-8003-3adcaebd9e2d req-4180ad7c-796d-4f09-94e4-d372874dc496 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.076955 Oct 28 09:54:20.797188 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 647/1059] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:54:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:54:21.351390 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell0:SELECT=8 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:21.359171 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-dbd74686-0be8-4c9d-9479-ff02449d1bc7 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:21.359900 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-dbd74686-0be8-4c9d-9479-ff02449d1bc7 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:21.367206 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbd74686-0be8-4c9d-9479-ff02449d1bc7 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:21.367726 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbd74686-0be8-4c9d-9479-ff02449d1bc7 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:21.368162 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbd74686-0be8-4c9d-9479-ff02449d1bc7 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:21.368880 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbd74686-0be8-4c9d-9479-ff02449d1bc7 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:21.369336 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbd74686-0be8-4c9d-9479-ff02449d1bc7 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:21.369787 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbd74686-0be8-4c9d-9479-ff02449d1bc7 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:21.596324 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-dbd74686-0be8-4c9d-9479-ff02449d1bc7 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:21.612838 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-dbd74686-0be8-4c9d-9479-ff02449d1bc7 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1314 microversion: 2.1 time: 0.263741 Oct 28 09:54:21.614409 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 648/1060] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:21 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1314 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:21.733184 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c8d5cd9f-0463-498c-a4f7-6dd0b68fc9f1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:21.733809 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c8d5cd9f-0463-498c-a4f7-6dd0b68fc9f1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] [instance: 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:21.740227 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8d5cd9f-0463-498c-a4f7-6dd0b68fc9f1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:21.740715 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8d5cd9f-0463-498c-a4f7-6dd0b68fc9f1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:21.741150 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8d5cd9f-0463-498c-a4f7-6dd0b68fc9f1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:21.741867 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8d5cd9f-0463-498c-a4f7-6dd0b68fc9f1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:21.742319 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8d5cd9f-0463-498c-a4f7-6dd0b68fc9f1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:21.742779 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c8d5cd9f-0463-498c-a4f7-6dd0b68fc9f1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:21.759860 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-c8d5cd9f-0463-498c-a4f7-6dd0b68fc9f1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] HTTP exception thrown: Instance 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 could not be found. Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c8d5cd9f-0463-498c-a4f7-6dd0b68fc9f1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Request method failure captured: Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 HTTP/1.1 Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6a18_O_azdMShsj3Nm0ny6pFFNTeNcnWwbWQIt80luJLjy3ygUUTinhpzL7rbLt6O6IdqbRKl2W7S1CY3DjK-SByNatrEvnc1AG81DddXEerqpniHgsIThMvjLF-mwBSGclwXfq6U4kqTXPeg6ksbmnOIbk4xU3ZT9Jy9k9ueCHyLY Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 1d832ea68ebb4c60b962d0cd76f787b8 Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-VolumesNegativeTest-1018315139 Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:54:21.761542 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:54:21.763259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1d832ea68ebb4c60b962d0cd76f787b8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1d832ea68ebb4c60b962d0cd76f787b8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1d832ea68ebb4c60b962d0cd76f787b8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1d832ea68ebb4c60b962d0cd76f787b8"}]}] Oct 28 09:54:21.763259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-VolumesNegativeTest-1018315139 Oct 28 09:54:21.763259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 1d832ea68ebb4c60b962d0cd76f787b8 Oct 28 09:54:21.763259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-VolumesNegativeTest-1018315139 Oct 28 09:54:21.763259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-VolumesNegativeTest-1018315139-project Oct 28 09:54:21.763259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:54:21.763259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:54:21.763259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: 4d088636881e45559699462268bd0411 Oct 28 09:54:21.763259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-VolumesNegativeTest-1018315139-project Oct 28 09:54:21.763259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:54:21.763259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 could not be found. Oct 28 09:54:21.763259 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3'} Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 could not be found. Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:54:21.764272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 could not be found. Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 could not be found. Oct 28 09:54:21.765863 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:21.767310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c8d5cd9f-0463-498c-a4f7-6dd0b68fc9f1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] Returning 404 to user: Instance 3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:54:21.767310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c8d5cd9f-0463-498c-a4f7-6dd0b68fc9f1 tempest-VolumesNegativeTest-1018315139 tempest-VolumesNegativeTest-1018315139-project] 10.222.0.25 "GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3" status: 404 len: 111 microversion: 2.1 time: 0.034813 Oct 28 09:54:21.767310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 649/1061] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:21 2022] GET /compute/v2.1/servers/3ac4b2ef-6841-4eda-ab6a-d1bb7d4cfdc3 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 09:54:22.643964 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3b946805-2a3f-4cdc-b38e-635b35786444 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:22.645926 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3b946805-2a3f-4cdc-b38e-635b35786444 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:22.656696 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b946805-2a3f-4cdc-b38e-635b35786444 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:22.657192 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b946805-2a3f-4cdc-b38e-635b35786444 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:22.657764 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b946805-2a3f-4cdc-b38e-635b35786444 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:22.659073 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b946805-2a3f-4cdc-b38e-635b35786444 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:22.659605 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b946805-2a3f-4cdc-b38e-635b35786444 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:22.660236 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b946805-2a3f-4cdc-b38e-635b35786444 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:22.927580 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3b946805-2a3f-4cdc-b38e-635b35786444 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:22.938050 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3b946805-2a3f-4cdc-b38e-635b35786444 tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1314 microversion: 2.1 time: 0.301467 Oct 28 09:54:22.939003 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 650/1062] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:22 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1314 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:22.957719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-62045d69-f2c4-482c-89a3-4d8cc3a6ce9d req-d178ae07-31ce-4f88-863b-e307908272c9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "352678cc-069f-4a32-b31b-7674427bbcb1", "name": "network-vif-plugged", "status": "completed", "tag": "efdf229e-e455-4624-b75f-3586f20bef0d"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:54:22.986304 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-62045d69-f2c4-482c-89a3-4d8cc3a6ce9d req-d178ae07-31ce-4f88-863b-e307908272c9 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:22.987009 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-62045d69-f2c4-482c-89a3-4d8cc3a6ce9d req-d178ae07-31ce-4f88-863b-e307908272c9 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:22.987553 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-62045d69-f2c4-482c-89a3-4d8cc3a6ce9d req-d178ae07-31ce-4f88-863b-e307908272c9 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:23.017170 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-62045d69-f2c4-482c-89a3-4d8cc3a6ce9d req-d178ae07-31ce-4f88-863b-e307908272c9 service nova] Creating event network-vif-plugged:efdf229e-e455-4624-b75f-3586f20bef0d for instance 352678cc-069f-4a32-b31b-7674427bbcb1 on hv-cinder-zed-3 Oct 28 09:54:23.027107 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-62045d69-f2c4-482c-89a3-4d8cc3a6ce9d req-d178ae07-31ce-4f88-863b-e307908272c9 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085594 Oct 28 09:54:23.028725 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 651/1063] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:54:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:54:23.984600 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9b29960e-7300-4cc2-8403-030708a7e48b tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:23.986585 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9b29960e-7300-4cc2-8403-030708a7e48b tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:24.000738 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9b29960e-7300-4cc2-8403-030708a7e48b tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:24.001801 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9b29960e-7300-4cc2-8403-030708a7e48b tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:24.002881 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9b29960e-7300-4cc2-8403-030708a7e48b tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:24.004442 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9b29960e-7300-4cc2-8403-030708a7e48b tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:24.005396 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9b29960e-7300-4cc2-8403-030708a7e48b tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:24.006252 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9b29960e-7300-4cc2-8403-030708a7e48b tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:24.221440 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-9b29960e-7300-4cc2-8403-030708a7e48b tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:24.231730 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9b29960e-7300-4cc2-8403-030708a7e48b tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1314 microversion: 2.1 time: 0.254734 Oct 28 09:54:24.232459 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 652/1064] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:23 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1314 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:25.278940 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-23975e0e-16a8-4715-b3a0-3e33f2d7c6ae tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:25.280159 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-23975e0e-16a8-4715-b3a0-3e33f2d7c6ae tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:25.289897 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-23975e0e-16a8-4715-b3a0-3e33f2d7c6ae tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:25.290640 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-23975e0e-16a8-4715-b3a0-3e33f2d7c6ae tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:25.291575 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-23975e0e-16a8-4715-b3a0-3e33f2d7c6ae tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:25.292573 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-23975e0e-16a8-4715-b3a0-3e33f2d7c6ae tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:25.293981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-23975e0e-16a8-4715-b3a0-3e33f2d7c6ae tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:25.294791 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-23975e0e-16a8-4715-b3a0-3e33f2d7c6ae tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:25.539981 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-23975e0e-16a8-4715-b3a0-3e33f2d7c6ae tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:25.550550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:25.575042 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-23975e0e-16a8-4715-b3a0-3e33f2d7c6ae tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1499 microversion: 2.1 time: 0.303372 Oct 28 09:54:25.586631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 653/1065] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:25 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1499 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:26.298935 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-83595b2e-1e00-43ce-8ded-b269d58822eb tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:26.299571 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-83595b2e-1e00-43ce-8ded-b269d58822eb tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:26.308520 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83595b2e-1e00-43ce-8ded-b269d58822eb tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:26.310335 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83595b2e-1e00-43ce-8ded-b269d58822eb tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:26.310902 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83595b2e-1e00-43ce-8ded-b269d58822eb tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:26.312019 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83595b2e-1e00-43ce-8ded-b269d58822eb tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:26.312971 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83595b2e-1e00-43ce-8ded-b269d58822eb tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:26.313717 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-83595b2e-1e00-43ce-8ded-b269d58822eb tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:26.356300 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-83595b2e-1e00-43ce-8ded-b269d58822eb tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Going to try to terminate instance {{(pid=127030) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:54:26.447751 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-83595b2e-1e00-43ce-8ded-b269d58822eb tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "DELETE /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 204 len: 0 microversion: 2.1 time: 0.152957 Oct 28 09:54:26.448878 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 654/1066] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:54:26 2022] DELETE /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 28 09:54:26.462527 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-757414dd-61f5-49c3-99a2-379a4ec8fa6c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:26.463411 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-757414dd-61f5-49c3-99a2-379a4ec8fa6c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:26.475718 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-757414dd-61f5-49c3-99a2-379a4ec8fa6c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:26.476713 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-757414dd-61f5-49c3-99a2-379a4ec8fa6c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:26.477753 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-757414dd-61f5-49c3-99a2-379a4ec8fa6c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:26.479310 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-757414dd-61f5-49c3-99a2-379a4ec8fa6c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:26.480555 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-757414dd-61f5-49c3-99a2-379a4ec8fa6c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:26.481514 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-757414dd-61f5-49c3-99a2-379a4ec8fa6c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:26.709066 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-757414dd-61f5-49c3-99a2-379a4ec8fa6c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:26.719518 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-757414dd-61f5-49c3-99a2-379a4ec8fa6c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1505 microversion: 2.1 time: 0.264886 Oct 28 09:54:26.720203 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 655/1067] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:26 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1505 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:54:27.747813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b871b564-4ef2-405b-af2e-5d984359b9db tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:27.748845 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b871b564-4ef2-405b-af2e-5d984359b9db tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:27.757561 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b871b564-4ef2-405b-af2e-5d984359b9db tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:27.758517 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b871b564-4ef2-405b-af2e-5d984359b9db tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:27.759372 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b871b564-4ef2-405b-af2e-5d984359b9db tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:27.760352 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b871b564-4ef2-405b-af2e-5d984359b9db tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:27.761113 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b871b564-4ef2-405b-af2e-5d984359b9db tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:27.764077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b871b564-4ef2-405b-af2e-5d984359b9db tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:28.001910 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-b871b564-4ef2-405b-af2e-5d984359b9db tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:28.023743 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b871b564-4ef2-405b-af2e-5d984359b9db tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1505 microversion: 2.1 time: 0.282853 Oct 28 09:54:28.025219 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 656/1068] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:27 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1505 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:28.805260 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-ef18c50d-2f71-4024-8e43-bc9ae22d1fe6 req-30758399-0cc3-4254-85fc-51f0ce728e0a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "352678cc-069f-4a32-b31b-7674427bbcb1", "name": "network-vif-deleted", "tag": "efdf229e-e455-4624-b75f-3586f20bef0d"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:54:28.826341 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-ef18c50d-2f71-4024-8e43-bc9ae22d1fe6 req-30758399-0cc3-4254-85fc-51f0ce728e0a service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:28.827238 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-ef18c50d-2f71-4024-8e43-bc9ae22d1fe6 req-30758399-0cc3-4254-85fc-51f0ce728e0a service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:28.828089 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-ef18c50d-2f71-4024-8e43-bc9ae22d1fe6 req-30758399-0cc3-4254-85fc-51f0ce728e0a service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:28.852898 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-ef18c50d-2f71-4024-8e43-bc9ae22d1fe6 req-30758399-0cc3-4254-85fc-51f0ce728e0a service nova] Creating event network-vif-deleted:efdf229e-e455-4624-b75f-3586f20bef0d for instance 352678cc-069f-4a32-b31b-7674427bbcb1 on hv-cinder-zed-3 Oct 28 09:54:28.862525 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-ef18c50d-2f71-4024-8e43-bc9ae22d1fe6 req-30758399-0cc3-4254-85fc-51f0ce728e0a service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064734 Oct 28 09:54:28.863609 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 657/1069] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:54:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:54:29.050765 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:29.052090 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:29.060272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:29.060905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:29.061809 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:29.062737 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:29.063336 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:29.063865 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:29.108637 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:29.109121 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:29.109566 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:29.110027 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lazy-loading 'fault' on Instance uuid 352678cc-069f-4a32-b31b-7674427bbcb1 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:54:29.157034 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a3c6fa6f402143b598108c6cb2aedf78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6737c612807a468797742417d59da976', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:54:29.172218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-cd0c0250-f147-42e0-87ab-9b864491e67c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 200 len: 1304 microversion: 2.1 time: 0.126297 Oct 28 09:54:29.178715 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 658/1070] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:29 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 1304 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:54:30.212470 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-dbca5175-dcb0-4486-af72-5c7c8c44616c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:54:30.213403 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-dbca5175-dcb0-4486-af72-5c7c8c44616c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] [instance: 352678cc-069f-4a32-b31b-7674427bbcb1] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:54:30.220941 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbca5175-dcb0-4486-af72-5c7c8c44616c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:30.221440 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbca5175-dcb0-4486-af72-5c7c8c44616c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:30.221884 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbca5175-dcb0-4486-af72-5c7c8c44616c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:30.222665 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbca5175-dcb0-4486-af72-5c7c8c44616c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:54:30.223123 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbca5175-dcb0-4486-af72-5c7c8c44616c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:54:30.223562 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-dbca5175-dcb0-4486-af72-5c7c8c44616c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:54:30.239374 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-dbca5175-dcb0-4486-af72-5c7c8c44616c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] HTTP exception thrown: Instance 352678cc-069f-4a32-b31b-7674427bbcb1 could not be found. Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-dbca5175-dcb0-4486-af72-5c7c8c44616c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Request method failure captured: Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 HTTP/1.1 Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6apzCi3qRWmIc52YfW5_muRZHyojKioFgPYDZZF22VzJ9G120yqUWTXaPaNW0CLEahGYHec3odWY-4qUYzAgdcuGH57jZpcLrj82y8oKpOFJV8odT0HMRXfmxBs5SDtTn2laSd6k9RWOUid3VGokRZ0CEt7mJcKO8KlV_7jV3PE_wA Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 6737c612807a468797742417d59da976 Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-VolumesActionsTest-1640912710 Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:54:30.241077 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:54:30.243745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/6737c612807a468797742417d59da976"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6737c612807a468797742417d59da976"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_6737c612807a468797742417d59da976"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/6737c612807a468797742417d59da976"}]}] Oct 28 09:54:30.243745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-VolumesActionsTest-1640912710 Oct 28 09:54:30.243745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 6737c612807a468797742417d59da976 Oct 28 09:54:30.243745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-VolumesActionsTest-1640912710 Oct 28 09:54:30.243745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-VolumesActionsTest-1640912710-project Oct 28 09:54:30.243745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:54:30.243745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:54:30.243745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: a3c6fa6f402143b598108c6cb2aedf78 Oct 28 09:54:30.243745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-VolumesActionsTest-1640912710-project Oct 28 09:54:30.243745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:54:30.243745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance 352678cc-069f-4a32-b31b-7674427bbcb1 could not be found. Oct 28 09:54:30.243745 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': '352678cc-069f-4a32-b31b-7674427bbcb1'} Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 352678cc-069f-4a32-b31b-7674427bbcb1 could not be found. Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:54:30.245343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 352678cc-069f-4a32-b31b-7674427bbcb1 could not be found. Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 352678cc-069f-4a32-b31b-7674427bbcb1 could not be found. Oct 28 09:54:30.247918 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:54:30.250617 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-dbca5175-dcb0-4486-af72-5c7c8c44616c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] Returning 404 to user: Instance 352678cc-069f-4a32-b31b-7674427bbcb1 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:54:30.250617 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-dbca5175-dcb0-4486-af72-5c7c8c44616c tempest-VolumesActionsTest-1640912710 tempest-VolumesActionsTest-1640912710-project] 10.222.0.25 "GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1" status: 404 len: 111 microversion: 2.1 time: 0.034091 Oct 28 09:54:30.250617 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 659/1071] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:54:30 2022] GET /compute/v2.1/servers/352678cc-069f-4a32-b31b-7674427bbcb1 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 09:55:09.135666 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3c71d410-4671-48f6-9d25-51e2381abe5e tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:55:09.137075 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3c71d410-4671-48f6-9d25-51e2381abe5e tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:09.139176 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:09.139838 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=38,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:09.147761 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c71d410-4671-48f6-9d25-51e2381abe5e tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:09.148251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c71d410-4671-48f6-9d25-51e2381abe5e tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:09.148681 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c71d410-4671-48f6-9d25-51e2381abe5e tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:09.149400 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c71d410-4671-48f6-9d25-51e2381abe5e tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:09.149845 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c71d410-4671-48f6-9d25-51e2381abe5e tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:09.150288 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3c71d410-4671-48f6-9d25-51e2381abe5e tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:09.179302 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-3c71d410-4671-48f6-9d25-51e2381abe5e tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lazy-loading 'flavor' on Instance uuid b96d9706-3980-4e3c-8a22-f18b9a7b4a16 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:55:09.265458 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3c71d410-4671-48f6-9d25-51e2381abe5e tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "POST /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16/action" status: 202 len: 0 microversion: 2.1 time: 0.133682 Oct 28 09:55:09.266218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 660/1072] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:55:09 2022] POST /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16/action => generated 0 bytes in 135 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 09:55:09.280070 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-708c6da1-566d-4441-b680-c6df0b5f90db tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:09.281481 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-708c6da1-566d-4441-b680-c6df0b5f90db tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:09.291853 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-708c6da1-566d-4441-b680-c6df0b5f90db tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:09.293045 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-708c6da1-566d-4441-b680-c6df0b5f90db tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:09.293724 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-708c6da1-566d-4441-b680-c6df0b5f90db tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:09.294690 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-708c6da1-566d-4441-b680-c6df0b5f90db tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:09.295204 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-708c6da1-566d-4441-b680-c6df0b5f90db tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:09.295647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-708c6da1-566d-4441-b680-c6df0b5f90db tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:09.733152 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-708c6da1-566d-4441-b680-c6df0b5f90db tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:09.742325 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-708c6da1-566d-4441-b680-c6df0b5f90db tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1569 microversion: 2.1 time: 0.470387 Oct 28 09:55:09.742947 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 661/1073] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:09 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1569 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:10.769384 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-78e4b631-2430-46a6-9edb-78d4bf97e48c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:10.770181 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-78e4b631-2430-46a6-9edb-78d4bf97e48c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:10.776993 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78e4b631-2430-46a6-9edb-78d4bf97e48c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:10.777493 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78e4b631-2430-46a6-9edb-78d4bf97e48c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:10.777928 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78e4b631-2430-46a6-9edb-78d4bf97e48c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:10.778671 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78e4b631-2430-46a6-9edb-78d4bf97e48c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:10.779125 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78e4b631-2430-46a6-9edb-78d4bf97e48c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:10.779552 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-78e4b631-2430-46a6-9edb-78d4bf97e48c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:10.953405 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-78e4b631-2430-46a6-9edb-78d4bf97e48c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:10.966870 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-78e4b631-2430-46a6-9edb-78d4bf97e48c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1569 microversion: 2.1 time: 0.200948 Oct 28 09:55:10.967868 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 662/1074] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:10 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1569 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:11.995844 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-62a429da-8471-4d21-bd4b-891e8ef7c94a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:11.996609 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-62a429da-8471-4d21-bd4b-891e8ef7c94a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:12.003885 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-62a429da-8471-4d21-bd4b-891e8ef7c94a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:12.004663 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-62a429da-8471-4d21-bd4b-891e8ef7c94a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:12.005393 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-62a429da-8471-4d21-bd4b-891e8ef7c94a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:12.006314 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-62a429da-8471-4d21-bd4b-891e8ef7c94a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:12.007007 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-62a429da-8471-4d21-bd4b-891e8ef7c94a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:12.007473 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-62a429da-8471-4d21-bd4b-891e8ef7c94a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:12.173325 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-62a429da-8471-4d21-bd4b-891e8ef7c94a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:12.181827 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-62a429da-8471-4d21-bd4b-891e8ef7c94a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1569 microversion: 2.1 time: 0.190253 Oct 28 09:55:12.183615 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 663/1075] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:11 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1569 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:13.208464 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-4de89892-05b2-4f27-93e4-92378266dbc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:13.209551 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-4de89892-05b2-4f27-93e4-92378266dbc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:13.217500 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4de89892-05b2-4f27-93e4-92378266dbc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:13.218104 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4de89892-05b2-4f27-93e4-92378266dbc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:13.218652 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4de89892-05b2-4f27-93e4-92378266dbc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:13.219406 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4de89892-05b2-4f27-93e4-92378266dbc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:13.219888 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4de89892-05b2-4f27-93e4-92378266dbc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:13.220343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-4de89892-05b2-4f27-93e4-92378266dbc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:13.379364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-4de89892-05b2-4f27-93e4-92378266dbc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:13.388747 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-4de89892-05b2-4f27-93e4-92378266dbc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1569 microversion: 2.1 time: 0.184794 Oct 28 09:55:13.389425 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 664/1076] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:13 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1569 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:14.416150 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-5aedfa12-75c3-4ff8-b0d4-9f7c059473dc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:14.416945 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-5aedfa12-75c3-4ff8-b0d4-9f7c059473dc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:14.425654 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5aedfa12-75c3-4ff8-b0d4-9f7c059473dc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:14.426146 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5aedfa12-75c3-4ff8-b0d4-9f7c059473dc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:14.426603 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5aedfa12-75c3-4ff8-b0d4-9f7c059473dc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:14.427372 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5aedfa12-75c3-4ff8-b0d4-9f7c059473dc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:14.428024 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5aedfa12-75c3-4ff8-b0d4-9f7c059473dc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:14.428710 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-5aedfa12-75c3-4ff8-b0d4-9f7c059473dc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:14.593608 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-5aedfa12-75c3-4ff8-b0d4-9f7c059473dc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:14.603956 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-5aedfa12-75c3-4ff8-b0d4-9f7c059473dc tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1569 microversion: 2.1 time: 0.191575 Oct 28 09:55:14.605270 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 665/1077] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:14 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1569 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:15.633715 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-fee10100-6d5a-4ac1-b04d-300994712508 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:15.634446 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-fee10100-6d5a-4ac1-b04d-300994712508 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:15.641367 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fee10100-6d5a-4ac1-b04d-300994712508 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:15.641859 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fee10100-6d5a-4ac1-b04d-300994712508 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:15.642298 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fee10100-6d5a-4ac1-b04d-300994712508 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:15.643067 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fee10100-6d5a-4ac1-b04d-300994712508 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:15.643524 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fee10100-6d5a-4ac1-b04d-300994712508 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:15.643948 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fee10100-6d5a-4ac1-b04d-300994712508 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:16.094953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-fee10100-6d5a-4ac1-b04d-300994712508 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:16.103356 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-fee10100-6d5a-4ac1-b04d-300994712508 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1569 microversion: 2.1 time: 0.473667 Oct 28 09:55:16.103905 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 666/1078] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:15 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1569 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:17.133517 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-6628ab47-04cd-4ea1-9d56-7b507a052271 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:17.134457 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-6628ab47-04cd-4ea1-9d56-7b507a052271 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:17.141292 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6628ab47-04cd-4ea1-9d56-7b507a052271 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:17.141777 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6628ab47-04cd-4ea1-9d56-7b507a052271 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:17.142201 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6628ab47-04cd-4ea1-9d56-7b507a052271 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:17.142950 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6628ab47-04cd-4ea1-9d56-7b507a052271 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:17.143393 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6628ab47-04cd-4ea1-9d56-7b507a052271 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:17.143829 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-6628ab47-04cd-4ea1-9d56-7b507a052271 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:17.324760 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-6628ab47-04cd-4ea1-9d56-7b507a052271 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:17.338007 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-6628ab47-04cd-4ea1-9d56-7b507a052271 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1569 microversion: 2.1 time: 0.208217 Oct 28 09:55:17.339482 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 667/1079] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:17 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1569 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:18.378039 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-69300598-2e60-4dd2-8997-1accef807f1a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:18.378767 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-69300598-2e60-4dd2-8997-1accef807f1a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:18.385623 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69300598-2e60-4dd2-8997-1accef807f1a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:18.386109 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69300598-2e60-4dd2-8997-1accef807f1a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:18.386583 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69300598-2e60-4dd2-8997-1accef807f1a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:18.387299 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69300598-2e60-4dd2-8997-1accef807f1a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:18.387747 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69300598-2e60-4dd2-8997-1accef807f1a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:18.388167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-69300598-2e60-4dd2-8997-1accef807f1a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:18.573569 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-69300598-2e60-4dd2-8997-1accef807f1a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:18.583698 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-69300598-2e60-4dd2-8997-1accef807f1a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1569 microversion: 2.1 time: 0.209475 Oct 28 09:55:18.584360 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 668/1080] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:18 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1569 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:55:19.538822 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 28 09:55:19.539442 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Fri Oct 28 09:55:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.25) !!! Oct 28 09:55:19.611092 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3aeede79-b6e6-486e-b90b-5bc4af7a7804 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:19.611651 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3aeede79-b6e6-486e-b90b-5bc4af7a7804 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:19.619090 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3aeede79-b6e6-486e-b90b-5bc4af7a7804 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:19.619581 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3aeede79-b6e6-486e-b90b-5bc4af7a7804 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:19.620018 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3aeede79-b6e6-486e-b90b-5bc4af7a7804 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:19.620731 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3aeede79-b6e6-486e-b90b-5bc4af7a7804 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:19.621182 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3aeede79-b6e6-486e-b90b-5bc4af7a7804 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:19.621604 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3aeede79-b6e6-486e-b90b-5bc4af7a7804 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:19.875420 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3aeede79-b6e6-486e-b90b-5bc4af7a7804 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:19.889106 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3aeede79-b6e6-486e-b90b-5bc4af7a7804 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1569 microversion: 2.1 time: 0.281601 Oct 28 09:55:19.890155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 669/1081] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:19 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1569 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:20.929391 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-990b212e-0b19-4a9e-b2a9-fbde6cff4f33 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:20.930133 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-990b212e-0b19-4a9e-b2a9-fbde6cff4f33 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:20.937611 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-990b212e-0b19-4a9e-b2a9-fbde6cff4f33 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:20.938127 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-990b212e-0b19-4a9e-b2a9-fbde6cff4f33 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:20.938632 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-990b212e-0b19-4a9e-b2a9-fbde6cff4f33 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:20.939443 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-990b212e-0b19-4a9e-b2a9-fbde6cff4f33 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:20.939961 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-990b212e-0b19-4a9e-b2a9-fbde6cff4f33 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:20.940426 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-990b212e-0b19-4a9e-b2a9-fbde6cff4f33 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:21.107475 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-990b212e-0b19-4a9e-b2a9-fbde6cff4f33 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:21.121051 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-990b212e-0b19-4a9e-b2a9-fbde6cff4f33 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1569 microversion: 2.1 time: 0.195665 Oct 28 09:55:21.121651 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 670/1082] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:20 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1569 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:22.145546 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-cbc29d41-c073-4021-9ca4-302dbbad97a4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:22.146301 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-cbc29d41-c073-4021-9ca4-302dbbad97a4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:22.153821 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbc29d41-c073-4021-9ca4-302dbbad97a4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:22.154306 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbc29d41-c073-4021-9ca4-302dbbad97a4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:22.154769 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbc29d41-c073-4021-9ca4-302dbbad97a4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:22.155483 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbc29d41-c073-4021-9ca4-302dbbad97a4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:22.155936 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbc29d41-c073-4021-9ca4-302dbbad97a4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:22.156361 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-cbc29d41-c073-4021-9ca4-302dbbad97a4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:22.361719 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-cbc29d41-c073-4021-9ca4-302dbbad97a4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:22.381491 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-cbc29d41-c073-4021-9ca4-302dbbad97a4 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1569 microversion: 2.1 time: 0.239204 Oct 28 09:55:22.383090 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 671/1083] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:22 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1569 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:23.422550 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-18d7ef64-3679-473e-b6bb-c34aee96b7fb tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:23.423888 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-18d7ef64-3679-473e-b6bb-c34aee96b7fb tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:23.434311 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-18d7ef64-3679-473e-b6bb-c34aee96b7fb tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:23.435191 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-18d7ef64-3679-473e-b6bb-c34aee96b7fb tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:23.435956 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-18d7ef64-3679-473e-b6bb-c34aee96b7fb tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:23.437281 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-18d7ef64-3679-473e-b6bb-c34aee96b7fb tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:23.438044 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-18d7ef64-3679-473e-b6bb-c34aee96b7fb tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:23.438797 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-18d7ef64-3679-473e-b6bb-c34aee96b7fb tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:23.606835 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-18d7ef64-3679-473e-b6bb-c34aee96b7fb tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:23.616309 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-18d7ef64-3679-473e-b6bb-c34aee96b7fb tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1570 microversion: 2.1 time: 0.199927 Oct 28 09:55:23.616892 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 672/1084] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:23 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1570 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:23.639318 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ea3662db-7307-4d34-8aac-65e49f3fadc5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:55:23.639915 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ea3662db-7307-4d34-8aac-65e49f3fadc5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:23.646287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ea3662db-7307-4d34-8aac-65e49f3fadc5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:23.646807 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ea3662db-7307-4d34-8aac-65e49f3fadc5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:23.647305 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ea3662db-7307-4d34-8aac-65e49f3fadc5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:23.648021 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ea3662db-7307-4d34-8aac-65e49f3fadc5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:23.648472 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ea3662db-7307-4d34-8aac-65e49f3fadc5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:23.648897 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ea3662db-7307-4d34-8aac-65e49f3fadc5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:23.692495 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ea3662db-7307-4d34-8aac-65e49f3fadc5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=127030) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 28 09:55:23.751778 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:23.782320 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ea3662db-7307-4d34-8aac-65e49f3fadc5 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "POST /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16/action" status: 202 len: 0 microversion: 2.1 time: 0.146954 Oct 28 09:55:23.782950 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 673/1085] 10.222.0.25 () {60 vars in 1278 bytes} [Fri Oct 28 09:55:23 2022] POST /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16/action => generated 0 bytes in 148 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 09:55:23.791010 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ae464661-c754-4171-b809-7b1b223140e3 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:23.791648 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ae464661-c754-4171-b809-7b1b223140e3 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:23.799279 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae464661-c754-4171-b809-7b1b223140e3 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:23.799279 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae464661-c754-4171-b809-7b1b223140e3 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:23.799563 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae464661-c754-4171-b809-7b1b223140e3 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:23.800095 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae464661-c754-4171-b809-7b1b223140e3 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:23.800579 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae464661-c754-4171-b809-7b1b223140e3 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:23.800996 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ae464661-c754-4171-b809-7b1b223140e3 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:24.024048 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ae464661-c754-4171-b809-7b1b223140e3 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:24.035066 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ae464661-c754-4171-b809-7b1b223140e3 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1518 microversion: 2.1 time: 0.247362 Oct 28 09:55:24.036262 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 674/1086] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:23 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1518 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:25.069220 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-124b61e3-6f82-496d-80f9-72462625a6b2 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:25.069963 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-124b61e3-6f82-496d-80f9-72462625a6b2 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:25.076685 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-124b61e3-6f82-496d-80f9-72462625a6b2 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:25.077222 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-124b61e3-6f82-496d-80f9-72462625a6b2 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:25.077781 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-124b61e3-6f82-496d-80f9-72462625a6b2 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:25.079114 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-124b61e3-6f82-496d-80f9-72462625a6b2 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:25.079724 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-124b61e3-6f82-496d-80f9-72462625a6b2 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:25.080207 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-124b61e3-6f82-496d-80f9-72462625a6b2 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:25.239818 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-124b61e3-6f82-496d-80f9-72462625a6b2 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:25.248750 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-124b61e3-6f82-496d-80f9-72462625a6b2 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1520 microversion: 2.1 time: 0.184368 Oct 28 09:55:25.249522 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 675/1087] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:25 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1520 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:26.275273 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-99e41a05-94fe-4ecd-8c52-47d5cac3c18a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:26.276328 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-99e41a05-94fe-4ecd-8c52-47d5cac3c18a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:26.282653 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-99e41a05-94fe-4ecd-8c52-47d5cac3c18a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:26.283138 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-99e41a05-94fe-4ecd-8c52-47d5cac3c18a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:26.283567 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-99e41a05-94fe-4ecd-8c52-47d5cac3c18a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:26.284261 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-99e41a05-94fe-4ecd-8c52-47d5cac3c18a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:26.284705 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-99e41a05-94fe-4ecd-8c52-47d5cac3c18a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:26.285115 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-99e41a05-94fe-4ecd-8c52-47d5cac3c18a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:26.535735 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-99e41a05-94fe-4ecd-8c52-47d5cac3c18a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:26.544496 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-99e41a05-94fe-4ecd-8c52-47d5cac3c18a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1576 microversion: 2.1 time: 0.273207 Oct 28 09:55:26.545079 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 676/1088] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:26 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1576 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:27.071163 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-f3350969-c783-45aa-b6b8-6e4af5b6ee27 req-4a135974-b047-4034-b3ab-80f53bc8d28b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b96d9706-3980-4e3c-8a22-f18b9a7b4a16", "name": "network-vif-unplugged", "status": "completed", "tag": "3d968631-497a-49af-abf4-297aa62504c8"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:55:27.081591 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-f3350969-c783-45aa-b6b8-6e4af5b6ee27 req-4a135974-b047-4034-b3ab-80f53bc8d28b service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:27.082152 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-f3350969-c783-45aa-b6b8-6e4af5b6ee27 req-4a135974-b047-4034-b3ab-80f53bc8d28b service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:27.082627 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-f3350969-c783-45aa-b6b8-6e4af5b6ee27 req-4a135974-b047-4034-b3ab-80f53bc8d28b service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:27.095682 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-f3350969-c783-45aa-b6b8-6e4af5b6ee27 req-4a135974-b047-4034-b3ab-80f53bc8d28b service nova] Creating event network-vif-unplugged:3d968631-497a-49af-abf4-297aa62504c8 for instance b96d9706-3980-4e3c-8a22-f18b9a7b4a16 on hv-cinder-zed-3 Oct 28 09:55:27.102033 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-f3350969-c783-45aa-b6b8-6e4af5b6ee27 req-4a135974-b047-4034-b3ab-80f53bc8d28b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035239 Oct 28 09:55:27.103229 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 677/1089] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:55:27 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:55:27.569309 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-3b05e369-a0ea-4433-bc4d-e420d42f824b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:27.569962 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-3b05e369-a0ea-4433-bc4d-e420d42f824b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:27.576674 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b05e369-a0ea-4433-bc4d-e420d42f824b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:27.577173 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b05e369-a0ea-4433-bc4d-e420d42f824b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:27.577604 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b05e369-a0ea-4433-bc4d-e420d42f824b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:27.578304 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b05e369-a0ea-4433-bc4d-e420d42f824b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:27.578782 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b05e369-a0ea-4433-bc4d-e420d42f824b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:27.579201 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-3b05e369-a0ea-4433-bc4d-e420d42f824b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:27.740917 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-3b05e369-a0ea-4433-bc4d-e420d42f824b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:27.749586 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-3b05e369-a0ea-4433-bc4d-e420d42f824b tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1576 microversion: 2.1 time: 0.185041 Oct 28 09:55:27.750343 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 678/1090] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:27 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1576 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:28.773492 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-753d45f6-efd4-47c4-b07e-3803b4ee6be9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:28.774218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-753d45f6-efd4-47c4-b07e-3803b4ee6be9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:28.782846 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-753d45f6-efd4-47c4-b07e-3803b4ee6be9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:28.783785 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-753d45f6-efd4-47c4-b07e-3803b4ee6be9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:28.783906 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-753d45f6-efd4-47c4-b07e-3803b4ee6be9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:28.784679 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-753d45f6-efd4-47c4-b07e-3803b4ee6be9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:28.785239 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-753d45f6-efd4-47c4-b07e-3803b4ee6be9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:28.785758 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-753d45f6-efd4-47c4-b07e-3803b4ee6be9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:28.933293 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-753d45f6-efd4-47c4-b07e-3803b4ee6be9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:28.941796 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-753d45f6-efd4-47c4-b07e-3803b4ee6be9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1576 microversion: 2.1 time: 0.172416 Oct 28 09:55:28.942373 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 679/1091] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:28 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1576 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:29.977326 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-22676a57-acfe-4509-9101-7037c38c1475 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:29.978062 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-22676a57-acfe-4509-9101-7037c38c1475 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:29.985295 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-22676a57-acfe-4509-9101-7037c38c1475 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:29.985788 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-22676a57-acfe-4509-9101-7037c38c1475 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:29.986234 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-22676a57-acfe-4509-9101-7037c38c1475 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:29.986999 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-22676a57-acfe-4509-9101-7037c38c1475 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:29.987459 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-22676a57-acfe-4509-9101-7037c38c1475 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:29.987885 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-22676a57-acfe-4509-9101-7037c38c1475 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:30.144660 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-22676a57-acfe-4509-9101-7037c38c1475 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:30.156906 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-22676a57-acfe-4509-9101-7037c38c1475 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1576 microversion: 2.1 time: 0.183237 Oct 28 09:55:30.157806 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 680/1092] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:29 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1576 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:31.181590 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9e305981-8b56-4131-939e-181c1372ea0d tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:31.182272 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-9e305981-8b56-4131-939e-181c1372ea0d tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:31.189512 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9e305981-8b56-4131-939e-181c1372ea0d tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:31.190071 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9e305981-8b56-4131-939e-181c1372ea0d tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:31.190555 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9e305981-8b56-4131-939e-181c1372ea0d tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:31.191328 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9e305981-8b56-4131-939e-181c1372ea0d tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:31.191803 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9e305981-8b56-4131-939e-181c1372ea0d tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:31.192358 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-9e305981-8b56-4131-939e-181c1372ea0d tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:31.352598 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-9e305981-8b56-4131-939e-181c1372ea0d tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:31.360896 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9e305981-8b56-4131-939e-181c1372ea0d tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1576 microversion: 2.1 time: 0.183010 Oct 28 09:55:31.361392 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 681/1093] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:31 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1576 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:32.384668 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-bf0a0df3-cff1-4bd9-abed-778a3edaff8c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:32.385428 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-bf0a0df3-cff1-4bd9-abed-778a3edaff8c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:32.391684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bf0a0df3-cff1-4bd9-abed-778a3edaff8c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:32.392139 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bf0a0df3-cff1-4bd9-abed-778a3edaff8c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:32.392572 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bf0a0df3-cff1-4bd9-abed-778a3edaff8c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:32.393266 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bf0a0df3-cff1-4bd9-abed-778a3edaff8c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:32.393714 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bf0a0df3-cff1-4bd9-abed-778a3edaff8c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:32.394132 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-bf0a0df3-cff1-4bd9-abed-778a3edaff8c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:32.559041 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-bf0a0df3-cff1-4bd9-abed-778a3edaff8c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:32.568089 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-bf0a0df3-cff1-4bd9-abed-778a3edaff8c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1576 microversion: 2.1 time: 0.187239 Oct 28 09:55:32.568820 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 682/1094] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:32 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1576 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:33.597085 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-89ee392f-659f-42d1-8c73-2bc96afd7f19 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:33.597682 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-89ee392f-659f-42d1-8c73-2bc96afd7f19 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:33.610953 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-89ee392f-659f-42d1-8c73-2bc96afd7f19 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:33.611444 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-89ee392f-659f-42d1-8c73-2bc96afd7f19 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:33.611876 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-89ee392f-659f-42d1-8c73-2bc96afd7f19 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:33.612581 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-89ee392f-659f-42d1-8c73-2bc96afd7f19 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:33.613026 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-89ee392f-659f-42d1-8c73-2bc96afd7f19 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:33.613443 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-89ee392f-659f-42d1-8c73-2bc96afd7f19 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:33.765157 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-89ee392f-659f-42d1-8c73-2bc96afd7f19 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:33.773364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-89ee392f-659f-42d1-8c73-2bc96afd7f19 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1576 microversion: 2.1 time: 0.180063 Oct 28 09:55:33.773944 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 683/1095] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:33 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1576 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:34.798895 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-c0442a6b-4510-4e49-abdd-8720a4d7bcf6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:55:34.799928 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-c0442a6b-4510-4e49-abdd-8720a4d7bcf6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:55:34.807746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c0442a6b-4510-4e49-abdd-8720a4d7bcf6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:34.808228 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c0442a6b-4510-4e49-abdd-8720a4d7bcf6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:34.808684 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c0442a6b-4510-4e49-abdd-8720a4d7bcf6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:34.809364 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c0442a6b-4510-4e49-abdd-8720a4d7bcf6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:34.809808 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c0442a6b-4510-4e49-abdd-8720a4d7bcf6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:34.810218 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-c0442a6b-4510-4e49-abdd-8720a4d7bcf6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:34.999544 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-c0442a6b-4510-4e49-abdd-8720a4d7bcf6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:55:35.008669 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-c0442a6b-4510-4e49-abdd-8720a4d7bcf6 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1563 microversion: 2.1 time: 0.214183 Oct 28 09:55:35.009030 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 684/1096] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:55:34 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1563 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:55:37.190517 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-6c6fec5b-bbaa-4dd7-942f-d832107fd488 req-c873ffc6-602f-4632-8779-a08e4fc11b44 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b96d9706-3980-4e3c-8a22-f18b9a7b4a16", "name": "network-vif-plugged", "status": "completed", "tag": "3d968631-497a-49af-abf4-297aa62504c8"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:55:37.201180 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-6c6fec5b-bbaa-4dd7-942f-d832107fd488 req-c873ffc6-602f-4632-8779-a08e4fc11b44 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:55:37.201655 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-6c6fec5b-bbaa-4dd7-942f-d832107fd488 req-c873ffc6-602f-4632-8779-a08e4fc11b44 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:55:37.202078 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-6c6fec5b-bbaa-4dd7-942f-d832107fd488 req-c873ffc6-602f-4632-8779-a08e4fc11b44 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:55:37.214849 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-6c6fec5b-bbaa-4dd7-942f-d832107fd488 req-c873ffc6-602f-4632-8779-a08e4fc11b44 service nova] Creating event network-vif-plugged:3d968631-497a-49af-abf4-297aa62504c8 for instance b96d9706-3980-4e3c-8a22-f18b9a7b4a16 on hv-cinder-zed-3 Oct 28 09:55:37.220506 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-6c6fec5b-bbaa-4dd7-942f-d832107fd488 req-c873ffc6-602f-4632-8779-a08e4fc11b44 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033773 Oct 28 09:55:37.221114 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 685/1097] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:55:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:56:51.557419 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 28 09:56:51.558186 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Fri Oct 28 09:56:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.25) !!! Oct 28 09:56:54.711577 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_api:SELECT=56,nova_api:UPDATE=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:56:54.713468 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG dbcounter [-] [127030] Writing DB stats nova_cell1:SELECT=65,nova_cell1:INSERT=1 {{(pid=127030) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:56:54.723061 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-b742ad6d-7a5b-47ad-b22f-0004fcf259b9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:56:54.724269 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b742ad6d-7a5b-47ad-b22f-0004fcf259b9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:56:54.738539 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b742ad6d-7a5b-47ad-b22f-0004fcf259b9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:56:54.739432 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b742ad6d-7a5b-47ad-b22f-0004fcf259b9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:56:54.740202 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b742ad6d-7a5b-47ad-b22f-0004fcf259b9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:56:54.741477 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b742ad6d-7a5b-47ad-b22f-0004fcf259b9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:56:54.742855 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b742ad6d-7a5b-47ad-b22f-0004fcf259b9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:56:54.743646 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-b742ad6d-7a5b-47ad-b22f-0004fcf259b9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:56:54.796472 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-b742ad6d-7a5b-47ad-b22f-0004fcf259b9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Going to try to terminate instance {{(pid=127030) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 09:56:54.902361 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-b742ad6d-7a5b-47ad-b22f-0004fcf259b9 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "DELETE /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 204 len: 0 microversion: 2.1 time: 0.194019 Oct 28 09:56:54.903326 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 686/1098] 10.222.0.25 () {58 vars in 1239 bytes} [Fri Oct 28 09:56:54 2022] DELETE /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 28 09:56:55.059803 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-fada6681-3710-45ad-8dbc-c04df9527494 req-bc02a001-e89b-4645-b5ba-d79d0906f496 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b96d9706-3980-4e3c-8a22-f18b9a7b4a16", "tag": "3d968631-497a-49af-abf4-297aa62504c8"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:56:55.071693 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-fada6681-3710-45ad-8dbc-c04df9527494 req-bc02a001-e89b-4645-b5ba-d79d0906f496 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:56:55.072294 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-fada6681-3710-45ad-8dbc-c04df9527494 req-bc02a001-e89b-4645-b5ba-d79d0906f496 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:56:55.073001 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-fada6681-3710-45ad-8dbc-c04df9527494 req-bc02a001-e89b-4645-b5ba-d79d0906f496 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:56:55.093994 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-fada6681-3710-45ad-8dbc-c04df9527494 req-bc02a001-e89b-4645-b5ba-d79d0906f496 service nova] Creating event network-changed:3d968631-497a-49af-abf4-297aa62504c8 for instance b96d9706-3980-4e3c-8a22-f18b9a7b4a16 on hv-cinder-zed-3 Oct 28 09:56:55.103552 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-fada6681-3710-45ad-8dbc-c04df9527494 req-bc02a001-e89b-4645-b5ba-d79d0906f496 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.197818 Oct 28 09:56:55.104558 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 687/1099] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:56:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 09:56:55.114449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-886b2b73-7cee-4e24-9398-d75600c7124a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:56:55.115746 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-886b2b73-7cee-4e24-9398-d75600c7124a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:56:55.127742 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-886b2b73-7cee-4e24-9398-d75600c7124a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:56:55.128607 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-886b2b73-7cee-4e24-9398-d75600c7124a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:56:55.129380 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-886b2b73-7cee-4e24-9398-d75600c7124a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:56:55.130670 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-886b2b73-7cee-4e24-9398-d75600c7124a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:56:55.131502 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-886b2b73-7cee-4e24-9398-d75600c7124a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:56:55.132263 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-886b2b73-7cee-4e24-9398-d75600c7124a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:56:55.379908 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-886b2b73-7cee-4e24-9398-d75600c7124a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:56:55.393167 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-886b2b73-7cee-4e24-9398-d75600c7124a tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1569 microversion: 2.1 time: 0.286534 Oct 28 09:56:55.394155 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 688/1100] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:56:55 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1569 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:56:56.432648 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-7a345caf-a7de-4e27-9f46-8f10bbc6770c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:56:56.433429 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-7a345caf-a7de-4e27-9f46-8f10bbc6770c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:56:56.440137 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a345caf-a7de-4e27-9f46-8f10bbc6770c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:56:56.440624 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a345caf-a7de-4e27-9f46-8f10bbc6770c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:56:56.441062 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a345caf-a7de-4e27-9f46-8f10bbc6770c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:56:56.441775 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a345caf-a7de-4e27-9f46-8f10bbc6770c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:56:56.442222 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a345caf-a7de-4e27-9f46-8f10bbc6770c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:56:56.442672 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-7a345caf-a7de-4e27-9f46-8f10bbc6770c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:56:56.657733 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-7a345caf-a7de-4e27-9f46-8f10bbc6770c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:56:56.671996 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-7a345caf-a7de-4e27-9f46-8f10bbc6770c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1569 microversion: 2.1 time: 0.243415 Oct 28 09:56:56.673718 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 689/1101] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:56:56 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1569 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:56:57.697740 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-ad71cb42-7f4e-4b5e-98db-6b3760168a02 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:56:57.698593 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-ad71cb42-7f4e-4b5e-98db-6b3760168a02 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:56:57.706287 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad71cb42-7f4e-4b5e-98db-6b3760168a02 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:56:57.706813 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad71cb42-7f4e-4b5e-98db-6b3760168a02 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:56:57.707251 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad71cb42-7f4e-4b5e-98db-6b3760168a02 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:56:57.707980 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad71cb42-7f4e-4b5e-98db-6b3760168a02 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:56:57.708495 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad71cb42-7f4e-4b5e-98db-6b3760168a02 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:56:57.708854 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-ad71cb42-7f4e-4b5e-98db-6b3760168a02 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:56:57.838761 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-ad71cb42-7f4e-4b5e-98db-6b3760168a02 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:56:57.849040 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-ad71cb42-7f4e-4b5e-98db-6b3760168a02 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1382 microversion: 2.1 time: 0.155883 Oct 28 09:56:57.849776 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 690/1102] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:56:57 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1382 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 09:56:57.862449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [req-30b31927-cc11-459f-804c-5edc648a889a req-1c81a731-5b44-4f5c-8abb-a7d764bf98d4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b96d9706-3980-4e3c-8a22-f18b9a7b4a16", "name": "network-vif-deleted", "tag": "3d968631-497a-49af-abf4-297aa62504c8"}]} {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 09:56:57.873501 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-30b31927-cc11-459f-804c-5edc648a889a req-1c81a731-5b44-4f5c-8abb-a7d764bf98d4 service nova] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:56:57.873984 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-30b31927-cc11-459f-804c-5edc648a889a req-1c81a731-5b44-4f5c-8abb-a7d764bf98d4 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:56:57.874439 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [req-30b31927-cc11-459f-804c-5edc648a889a req-1c81a731-5b44-4f5c-8abb-a7d764bf98d4 service nova] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:56:57.891535 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.compute.server_external_events [req-30b31927-cc11-459f-804c-5edc648a889a req-1c81a731-5b44-4f5c-8abb-a7d764bf98d4 service nova] Creating event network-vif-deleted:3d968631-497a-49af-abf4-297aa62504c8 for instance b96d9706-3980-4e3c-8a22-f18b9a7b4a16 on hv-cinder-zed-3 Oct 28 09:56:57.897924 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [req-30b31927-cc11-459f-804c-5edc648a889a req-1c81a731-5b44-4f5c-8abb-a7d764bf98d4 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038912 Oct 28 09:56:57.899123 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 691/1103] 10.222.0.25 () {64 vars in 1317 bytes} [Fri Oct 28 09:56:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 09:56:58.874961 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:56:58.875733 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:56:58.883729 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:56:58.884383 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:56:58.884867 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:56:58.885631 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:56:58.886139 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:56:58.886647 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:56:58.922216 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:56:58.922748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:56:58.923444 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:56:58.923939 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.objects.instance [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lazy-loading 'fault' on Instance uuid b96d9706-3980-4e3c-8a22-f18b9a7b4a16 {{(pid=127030) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 09:56:58.985148 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.policy [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f14c3f34010b41ef9be2b9fd2aa98cd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '937954dbeac043c29259ada2924c4953', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127030) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 09:56:58.993652 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-00ae5493-3ab9-4937-9985-1d1e6b9df9a1 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 200 len: 1177 microversion: 2.1 time: 0.123354 Oct 28 09:56:58.994247 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 692/1104] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:56:58 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 1177 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 09:57:00.017316 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-fec32ff0-f0ec-4992-95b6-1161f4a09cc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:57:00.018113 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.compute.api [None req-fec32ff0-f0ec-4992-95b6-1161f4a09cc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] [instance: b96d9706-3980-4e3c-8a22-f18b9a7b4a16] Fetching instance by UUID {{(pid=127030) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 09:57:00.024988 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fec32ff0-f0ec-4992-95b6-1161f4a09cc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:57:00.025472 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fec32ff0-f0ec-4992-95b6-1161f4a09cc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:57:00.025900 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fec32ff0-f0ec-4992-95b6-1161f4a09cc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:57:00.026748 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fec32ff0-f0ec-4992-95b6-1161f4a09cc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Acquiring lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 09:57:00.027240 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fec32ff0-f0ec-4992-95b6-1161f4a09cc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 09:57:00.027701 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG oslo_concurrency.lockutils [None req-fec32ff0-f0ec-4992-95b6-1161f4a09cc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Lock "7c09c1b6-7514-40b3-bb3f-05d78565b59d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127030) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 09:57:00.042449 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.wsgi [None req-fec32ff0-f0ec-4992-95b6-1161f4a09cc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] HTTP exception thrown: Instance b96d9706-3980-4e3c-8a22-f18b9a7b4a16 could not be found. Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-fec32ff0-f0ec-4992-95b6-1161f4a09cc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Request method failure captured: Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: request: GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 HTTP/1.1 Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept: application/json Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Accept-Encoding: identity Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Connection: close Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Length: 0 Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Content-Type: application/json Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Host: 10.222.0.25 Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: Openstack-System-Scope: None Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: User-Agent: python-urllib3/1.26.12 Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Auth-Token: gAAAAABjW6Y_DfoNAWi-gIshCfIr6ecTM72q7pWg51glrrReBSXDTlGd0NxzyXW4YiRLHHtMfwTZJwmF5aRtqJqeh7_qx2jlWUZA0ZMlqhEmUY1NsJVzgYcqGFJC75R_QjjbV8tNoYytkEd2pCbxR2SyIT28qVu2NiKEwdR2--ljCCF1yuhWBTU Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Id: None Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Domain-Name: None Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Identity-Status: Confirmed Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Is-Admin-Project: True Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Id: default Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Domain-Name: Default Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Id: 937954dbeac043c29259ada2924c4953 Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Project-Name: tempest-TestShelveInstance-695044665 Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Role: member,reader Oct 28 09:57:00.044144 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Roles: member,reader Oct 28 09:57:00.046904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/937954dbeac043c29259ada2924c4953"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/937954dbeac043c29259ada2924c4953"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_937954dbeac043c29259ada2924c4953"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/937954dbeac043c29259ada2924c4953"}]}] Oct 28 09:57:00.046904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant: tempest-TestShelveInstance-695044665 Oct 28 09:57:00.046904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Id: 937954dbeac043c29259ada2924c4953 Oct 28 09:57:00.046904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-Tenant-Name: tempest-TestShelveInstance-695044665 Oct 28 09:57:00.046904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User: tempest-TestShelveInstance-695044665-project Oct 28 09:57:00.046904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Id: default Oct 28 09:57:00.046904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Domain-Name: Default Oct 28 09:57:00.046904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Id: f14c3f34010b41ef9be2b9fd2aa98cd9 Oct 28 09:57:00.046904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: X-User-Name: tempest-TestShelveInstance-695044665-project Oct 28 09:57:00.046904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: method: > Oct 28 09:57:00.046904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: exception: Instance b96d9706-3980-4e3c-8a22-f18b9a7b4a16 could not be found. Oct 28 09:57:00.046904 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: action_args: {'id': 'b96d9706-3980-4e3c-8a22-f18b9a7b4a16'} Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b96d9706-3980-4e3c-8a22-f18b9a7b4a16 could not be found. Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 09:57:00.049094 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b96d9706-3980-4e3c-8a22-f18b9a7b4a16 could not be found. Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b96d9706-3980-4e3c-8a22-f18b9a7b4a16 could not be found. Oct 28 09:57:00.051638 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: ERROR nova.api.openstack.wsgi Oct 28 09:57:00.055211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-fec32ff0-f0ec-4992-95b6-1161f4a09cc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Returning 404 to user: Instance b96d9706-3980-4e3c-8a22-f18b9a7b4a16 could not be found. {{(pid=127030) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 09:57:00.055211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-fec32ff0-f0ec-4992-95b6-1161f4a09cc0 tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16" status: 404 len: 111 microversion: 2.1 time: 0.032364 Oct 28 09:57:00.055211 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 693/1105] 10.222.0.25 () {58 vars in 1236 bytes} [Fri Oct 28 09:57:00 2022] GET /compute/v2.1/servers/b96d9706-3980-4e3c-8a22-f18b9a7b4a16 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 09:57:00.230770 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: DEBUG nova.api.openstack.wsgi [None req-9767db84-5a62-4cb0-a1d9-e47b34512c4c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] Calling method '.version_select at 0x7f271a5598b0>' {{(pid=127030) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 09:57:00.260014 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: INFO nova.api.openstack.requestlog [None req-9767db84-5a62-4cb0-a1d9-e47b34512c4c tempest-TestShelveInstance-695044665 tempest-TestShelveInstance-695044665-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-85316997" status: 202 len: 0 microversion: 2.1 time: 0.034789 Oct 28 09:57:00.260738 dv-cinder-zed-3-iscsi devstack@n-api.service[127030]: [pid: 127030|app: 0|req: 694/1106] 10.222.0.25 () {58 vars in 1248 bytes} [Fri Oct 28 09:57:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-85316997 => generated 0 bytes in 37 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 09:57:11.168739 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 28 09:57:11.169514 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Fri Oct 28 09:57:11 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.25) !!! Oct 28 09:58:39.550483 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 28 09:58:39.551565 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Fri Oct 28 09:58:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.25) !!! Oct 28 10:00:11.569341 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 28 10:00:11.570107 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Fri Oct 28 10:00:11 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.25) !!! Oct 28 10:00:31.180846 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 28 10:00:31.181549 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Fri Oct 28 10:00:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.25) !!! Oct 28 10:01:59.562589 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 28 10:01:59.563336 dv-cinder-zed-3-iscsi devstack@n-api.service[127029]: Fri Oct 28 10:01:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.25) !!!